I recently received this info:
Did you know as of April 11th there was no longer a need to manually submit your sitemap to search engines. Last fall, the major search engines agreed on a sitemaps format. You can now add a simple line to your robots.txt file and let the engines know where your sitemap document resides on your site
So I duly created a sitemap with an online generator and .. I don't appear to have a robots.txt file.

Can anyone give me the code? And where to put it? In the head? or the body?

This is the line I need to add to the robots file:

http://www.softwarecybersleuth.co.uk/sitemap.xml

Any help greatly appreciated :thumbsup:

Have you been here?

http://www.robotstxt.org/

I recently received this info:

So I duly created a sitemap with an online generator and .. I don't appear to have a robots.txt file.

Can anyone give me the code? And where to put it? In the head? or the body?

This is the line I need to add to the robots file:

http://www.softwarecybersleuth.co.uk/sitemap.xml

Any help greatly appreciated :thumbsup:

Biggest load of rubbish out!!, i am an SEO specialist and do it for a living and can tell you that you do NOT NEED a sitemap on your site, as long as its well laid out then the search engines can spider the entire site no problem

Biggest load of rubbish out!!, i am an SEO specialist and do it for a living and can tell you that you do NOT NEED a sitemap on your site, as long as its well laid out then the search engines can spider the entire site no problem

A little strong..

http://www.google.co.uk/search?hl=en&q=sitemap.xml+robots.txt&meta=

Although it seems that there are still some wrinkles.

Yes ok, perhaps it was a bit strong, but believe me, i have met some of the supposed best SEO "experts" in this country, and its amazing what crap they talk. I have been doing SEO for a lot of websites now for the last few years and it annoys me the amount of rubbish that gets posted and people believe.
It makes my job that much harder at the start as these people come along with these pre-conceptions that i have to disprove

Biggest load of rubbish out!!, i am an SEO specialist and do it for a living and can tell you that you do NOT NEED a sitemap on your site, as long as its well laid out then the search engines can spider the entire site no problem

Handy for some things though?

For instance if you don't want any site images ending up on google images search, you can exclude said images from being crawled in your sitemap :)

cgksheff and joep - thanks :)

Sorted - simplest robots.txt file ever.. (from here) http://www.gnc-web-creations.com/creating_robotstxt_file.htm#What%20is%20a%20simple %20robots.txt%20file

User-agent: *
Disallow:
SITEMAP:http://www.softwarecybersleuth.co.uk/sitemap.xml

And a robots.txt syntax checker if it's any use to anyone.
http://tool.motoricerca.info/robots-checker.phtml