User-agent: bingbot Crawl-delay: 2 User-agent: yandex Crawl-delay: 2 User-agent: * Disallow: /cron/