Parser not getting sitemap from robots.txt

[TheUltimateCookie]

For this URL - https://www.zendesk.com/robots.txt, it contains a sitemap, but the data is not getting collected.

Here's my code

resp, err := http.Get("https://www.zendesk.com/robots.txt")
if err != nil {
    log.Fatal(err)
}

robots, err := robotstxt.FromResponse(resp)
resp.Body.Close()
if err != nil {
    log.Fatal(err)
}

Collected Info &{map[] true false []}

Please advice

[temoto]

@TheUltimateCookie please try logging response code and body. Likely, you are hitting CDN (Cloudflare) captcha. Isn't it ironic, that Zendesk thinks you must be human to read robots.txt? :-)

[TheUltimateCookie]

@temoto Oh yes, just noticed that I'm getting 403 for both their robots.txt and their sitemap. Only accessible if I visit as human. Thank you for your time