To avoid undesirable articles from the look for indexes, webmasters can instruct spiders never to crawl certain documents or directories in the normal robots.txt file in the foundation directory from the domain. Also, a web page is usually explicitly excluded from a online search engine's database by making use of https://news.rafeeg.ae