Tag Archives: seo

A Side Note On WordPress, SEO, sitemap.xml and robots.txt

Last week, I learned that WordPress doesn’t ship with a default robots.txt.

  • this is the default file that search engine crawlers parse to see what resources and URL patterns that it allowed and not allowed to crawl; it’s step 1 in every search engine optimization (SEO) guide.

I guess I just stupidly assumed that it was included in WP. Anyways, I thought it to be fair to tell everyone that if you are using WordPress and you care how your site shows up in search results, you should generate a robots.txt and a sitemap.xml.
Continue reading