FREE Newsletter

Discover Top Internet Marketing Strategies! Free ebooks, marketing insider tips, & secret member-only perks... sign up now!

Getting Started

>	Free Newsletter
>	How to Build a Website
>	Getting Started
>	Business Ideas
>	Affiliate Marketing
>	SEO & Traffic
>	Ezines & Newsletters
>	Blogging & RSS
>	Press Release & Publicity
>	Writing & Copywriting
>	Online Auctions
>	Online Business Articles
>	Direct Sales Center
>	Work from Home
>	Service Business Articles
>	Small Business Bookkeeping
>	WAHM Success Stories

Our Services:

>	Build a Website
>	Cheap Domains
>	WAHM Mentoring
>	Reprint Articles for Moms
>	Website Content for Moms
>	Advertising

Networking & Community:

>	Work at Home Moms Message Boards
>	Work at Home Blog
>	Work at Home EXPERT Blogs
>	Free Membership
>	Breast Cancer Awareness

Directories

>	Business Opportunity Listings
>	Direct Sales Center
>	Internet Marketing Tools
>	Online Classified Ads
>	WAHM Mall

Just for Moms

>	Time Management & Organization
>	Health & Wellbeing
>	Money & Savings
>	Parenting & Family
>	Diet & Fitness
>	Cooking & Recipes

Search Engine& Web Building Information > Techie Stuff > Create Robots Txt

Create Robots Txt

A robots txt is a text file uploaded to your website server to keep compliant search engine robots from indexing particular pages of your website. Create a robots txt file like this

User-agent: Search Engine Robot Name
Disallow: File or Directory

Ex.

User-agent: Googlebot
Disallow: /adcampaign1.html

User-agent: *
Disallow: /adcampaign1.html

The * is used to cover all compliant search engine robots.

Common Robot Names

Search engine robots follow links throughout the Internet to crawl through all the websites on the world wide web. Here's a list of top search engine robot names:

Google:	googlebot
Ask Jeeves:	Teoma
Alta Vista / Yahoo / All the Web / Inktomi	Slurp

Check Your Robots txt on Search Engine World's Robots.txt ValidatorCheck Your Robots txt on Search Engine World's Robots.txt Validator

Or Use This Free Robots.txt Generator

More Search Engine Information: Highly Recommended!

Check Aaron Wall's SEO Book. Aaron's book doesn't leave any stones unturned. He covers everything you need to know about getting traffic to your website and legitimately ranking well with the search engines. His book covers all the nitty gritty details in an easy-to-understand way.

Click here to learn how...