Stop Google from indexing pages

Below are multiple ways you can stop Google from indexing pages or ask google to remove pages already indexed by google:

I think there are many occasions where you may want to exclude a website or portion of a site from search engine crawling and indexing.

There are several ways to prevent Google, Yahoo!, Bing or Ask from indexing a site’s pages.

• Using robots.txt file :A robots.txt file restricts access to your site by search engine robots that crawl the web. These bots are automated, and before they access pages of a site, they check to see if a robots.txt file exists that prevents them from accessing certain pages. (All respectable robots will respect the directives in a robots.txt file, although some may interpret them differently. However, a robots.txt is not enforceable, and some spammers and other troublemakers may ignore it. For this reason, we recommend password protecting confidential information.) You need a robots.txt file only if your site includes content that you don’t want search engines to index. If you want search engines to index everything in your site, you don’t need a robots.txt file (not even an empty one).

• Using the robots meta tag: To entirely prevent a page’s contents from being listed in the Google web index even if other sites link to it, use a noindex meta tag. As long as Googlebot fetches the page, it will see the noindex meta tag and prevent that page from showing up in the web index. <meta name=”robots” content=”noindex”>

• Using the URL removal request tool in Google Webmaster Tools
• If the page no longer exists, make sure that the server returns a 404 (Not Found) or 410 (Gone) HTTP status code. This will tell Google that the page is gone and that it should no longer appear in search results. you can even use 301 permanent redirects
• Password protect sensitive content: Sensitive content is usually protected by requiring visitors to enter a username and password. Such secure content won’t be crawled by search engines. Passwords can be set at the web server level or at the application level. For server level logon setup, consult the Apache Authentication Documentation or the Microsoft IIS documentation.
• Don’t link to pages you want to keep out of search engines