Added possibility to change HTTP request headers e.g. User-Agent

benbalter / sitemap-parser

Ruby Gem to parse sitemaps.org compliant sitemaps

MIT License

29 stars 46 forks source link

Closed sscherban closed 4 months ago

sscherban commented 11 months ago

Sometime access to sitemap.xml is not allowed for specific user agents and we need to redefine it. This one line patch give possibility to do this.

ishields commented 4 months ago

Any idea why this is failing?