# http://www.robotstxt.org/wc/exclusion.html#robotstxt User-agent: * Disallow: /cgi-bin/ Disallow: /manual/ # http://validator.w3.org/docs/checklink.html#bot User-Agent: W3C-checklink Disallow: User-Agent: * Disallow: /english User-Agent: * Disallow: /admin User-Agent: * Disallow: /DATA/~recht User-Agent: * Disallow: /getData.php User-Agent: * Disallow: /DATA/projektseiten User-Agent: * Disallow: /754.php