# robots.txt for cnet.de # All robots User-agent: * Disallow: /ajax/ # Disallow Internet Archive Wayback Machine User-agent: ia_archiver Disallow: /