#crawlers User-agent: GPTBot Crawl-delay: 10 Disallow: /?eventDate= Disallow: /page/ Disallow: /events/list/page/ User-agent: Amazonbot Crawl-delay: 10 Disallow: /?eventDate= Disallow: /page/ User-agent: SemrushBot Crawl-delay: 20 User-agent: AhrefsBot Crawl-delay: 20 User-agent: meta-externalagent Crawl-delay: 30 User-agent: facebookexternalhit Crawl-delay: 30 User-agent: CCBot Crawl-delay: 30 User-agent: TerraCotta Crawl-delay: 30