Sitemap: https://www.301bg.com/sitemap.xml Sitemap: https://www.301bg.com/sitemap2.xml # go away User-agent: * User-Agent: Googlebot Allow: / User-Agent: Googlebot-Mobile Allow: / User-agent: Mediapartners-Google Disallow: User-Agent: Bingbot Allow: / User-Agent: Bing Allow: / User-Agent: sp_auditbot Allow: / Crawl-Delay: 30 User-agent: AhrefsBot Crawl-Delay: 30 User-Agent: MJ12bot Crawl-Delay: 30 User-agent: SeznamBot Request-rate: 5/90 User-agent: BaiduSpider Disallow: / User-Agent: Mail.RU_Bot Disallow: / User-agent: BUbiNG Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: CCbot Disallow: / User-agent: GPTbot Disallow: / User-agent: OmgiliBot Disallow: / User-agent: Yandex Disallow: / User-agent: * Crawl-delay: 30