Subscribe to Brisbane Search Engine Optimization » SEO « Training - Advice - Tools - ResourcesNews FeedSubscribe to Brisbane Search Engine Optimization » SEO « Training - Advice - Tools - ResourcesComments
Browse > Home / Latest SEO News / Blocking Complicated URLs with Robots.txt

Blocking Complicated URLs with Robots.txt

November 7, 2008 by admin  
Filed under Latest SEO News

Leave a comment

Blocking Complicated URLs with Robots.txt
If you have a large web site you might have some content that you do not want the search engines to index — perhaps for duplicate content reasons or you simply don t want someone casually stumbling across it. You know you can use robots.txt but what if you need to block thousands of pages or block only certain files within a folder This article will explain some of the more advanced uses of robots.txt. You will even learn how to block dynamic pages …

Share and Enjoy:
  • Digg
  • Sphinn
  • del.icio.us
  • Facebook
  • Mixx
  • Google
  • Live
  • YahooMyWeb
  • E-mail this story to a friend!

Tags: , , , , , ,
rss icon RSS: Click on a tag, then click here for the RSS Feed

Latest Posts

Defining Page Rank Based On Popularity by admin on June 26th, 2008
The web search technology offered by Google is often the technology of choice of the world’s leading portals and websites.

Phoenix Mars Mission - Blogs by admin on September 15th, 2008
Phoenix Mars Mission - BlogsR esearchers and students take on the mission and the world of space exploration.

Underreported Stories from Search Engine Strategies Chicago 2008 by admin on December 14th, 2008
Underreported Stories from Search Engine Strategies Chicago 2008 All week long,...

Free Link Exchange - Free Links - Add Your Link - Reciprocal Link by admin on December 12th, 2008
Free Link Exchange - Free Links - Add Your Link - Reciprocal Link Free link exchange - Link...

Internet Marketing Specialist: the Key to getting money on your Business Website by admin on January 5th, 2009
Internet Marketing Specialist: the Key to getting money on your Business Website As the market competition...

Latest Comments

On An Introduction to Google Search Quality by fractionation on May 29th, 2008
[...] Our job is clear: A few hundreds of millions of times a day people will ask Google questions, and...

On JP Morgan Lowers Search Ad Growth Estimates by JP Morgan Lowers Search Ad Growth Estimates | Pick & Shovel News on November 26th, 2008
[...] continued here Powered by SmartRSS convey_url = [...]...

On Learning to Crawl: an Investigation of the Personal Web Crawler by SEO White Hats on October 31st, 2008
Good point! If any of your visitors are interesting in learning more about search engine optimizaiton, check out our...

On Search Engine Optimization & Internet Marketing, Phoenix Arizona by Posts about SEO as of December 12, 2008 | The Lessnau Lounge on December 13th, 2008
[...] a new blog and started setting it up but I didn’t manage to reach 5 posts or do any...

On Warren Buffett’s Subprime SEO by webmasters article directory on November 2nd, 2008
[...] Research and Analytics :: N.. Building A Website That Will Be Successful | The Website Doctor Warren BuffettÂ’s Subprime...

Comments

Feel free to leave a comment...
and oh, if you want a pic to show with your comment, go get a gravatar!