Blocking Complicated URLs with Robots.txt

November 7, 2008 by admin  
Filed under Latest SEO News

Blocking Complicated URLs with Robots.txt
If you have a large web site you might have some content that you do not want the search engines to index — perhaps for duplicate content reasons or you simply don t want someone casually stumbling across it. You know you can use robots.txt but what if you need to block thousands of pages or block only certain files within a folder This article will explain some of the more advanced uses of robots.txt. You will even learn how to block dynamic pages …

Share and Enjoy:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • Live
  • YahooMyWeb
  • E-mail this story to a friend!
Popular Posts

HRA Wrap-up 254 - Corie Whalen in the News! by admin on April 17th, 2009
HRA Wrap-up 254 - Corie Whalen in the News! What an exciting few weeks it's been!.

CitySquares Incorporates Facebook Connect by admin on August 9th, 2009
CitySquares Incorporates Facebook Connect Fast-growing local search engine CitySquares has incorporated...

Google Introduces Expandable Ads by admin on March 6th, 2009
Google Introduces Expandable Ads Earlier today, Google updated its official AdWords blog with the announcement of the introduction...

SEO (System Engine Optimizer) - Don�t miss it by admin on December 22nd, 2008
SEO (System Engine Optimizer) - Don�t miss it Powerful new tool will give you the...

Online Christmas Spending Surge 2009: What to Expect by admin on December 18th, 2009
Online Christmas Spending Surge 2009: What to Expect The Christmas spirit is in the air, and this...

Featured Comments

On How to Develop Link Exchange Strategies by linking strategy on June 4th, 2008
[...] can take up. But you may wonder how that would help you or your website in getting ahead of...

On Keyword Research | InlineSEO.com by justin brooke on January 28th, 2009
one of the secrets of successful websites is that they are making use of keywords that are being searched by...

On Facebook's FriendFeed Acquisition Could Have Implications for Search (Yes, Search) by Harsh Agrawal on August 12th, 2009
This is an important move and good news for Friendfeed, because despite of great effort in past few days ,...

On How to Prevent Affiliates from Driving Up PPC Costs - Advanced Forum Thread of the Week by How to Prevent Affiliates from Driving Up PPC Costs - Advanced … on December 3rd, 2008
[...] read the full article Categories : Google Adwords Comments [...]...

On No Copy on Home Page - Forum Thread of the Week by Choozyphoms on August 14th, 2009
This look interesting,so far. If there's anyone else here, let me know. Oh, and yes I'm a real person LOL. Later,...

Comments

Feel free to leave a comment...
and oh, if you want a pic to show with your comment, go get a gravatar!