The Robots Tag

If you have pages that you specifically DO NOT want spiders to see, you should use a robots tags to specify it. You may wonder why would you want to make sure a page doesn't get indexed. Consider private pages on your site, login pages, etc.

Along with your other META tags, place the following code:

<meta name="robots" content="noindex,nofollow">

or

<meta name="robots" content="none">

If you want to make sure the page does not get indexed, but the spider can follow any links it finds, you can use this tag:

<meta name="robots" content="noindex,follow">