# www.robotstxt.org # code.google.com/web/controlcrawlindex User-Agent: * Disallow: /cgi-bin/ Disallow: /webstat/ Disallow: /cache/ Disallow: /core/ Disallow: /minify/ Disallow: /settings/ Allow: / sitemap: /sitemap.xml