search

GateNLP / ultimate-sitemap-parser

Ultimate Website Sitemap Parser
https://mediacloud.org/
Other
181 stars 64 forks source link

Authentication Method for Secured Sites? #34

Open ma26yank opened 2 years ago

ma26yank commented 2 years ago

Hi, First of all, Thank you for building such a fantastic library for searching sitemaps through a website. There is a enhancement though which I would like to propose.

I have noticed for one of the site when I was trying the same thing, it asks for authentication via HttpNtlmAuth.

I was wondering if such arguments can be passed along with url to bypass authentication?

thanks