Prevent Search Engine Access To Website Folder

To Prevent search engine from indexing your site or URL:

  • Create robots.txt with the following content:
User-agent: *
Disallow: /

Place the file in the directory to instruct search engine to stop crawling (not all search engine will follow this rule).

  • If you are running a web server that support .htaccess method:
Options -Indexes

If your content has already been indexed by Google, login to https://search.google.com/search-console to prevent the search engine from indexing a domain or a URL (and sub folders).