# Web crawlers do not have a legal obligation to respect robots.txt (but it's nice if they do).

User-agent: *
Crawl-delay: 2
Disallow: /*/cgi-bin