# Make changes for all web spiders User-agent: * Disallow: /includes Disallow: /admin Disallow: /media Disallow: /stats Disallow: /stats Disallow: /webapp Disallow: /_db_backups Disallow: /wp