User-Agent: *
Allow: /


User-agent: yahoo-mmcrawler
User-agent: psbot
User-agent: WebReaper
User-agent: WebCopier
User-agent: Offline Explorer
User-agent: HTTrack
User-agent: Microsoft.URL.Control
User-agent: EmailCollector
Disallow: /