search

Lookyloo / pysecuritytxt

Tries to get and parse .well-known/security.txt from a domain
BSD 3-Clause "New" or "Revised" License
8 stars 2 forks source link

Some sites block requests made with the default user agent #6

Closed omstaendlig closed 1 year ago

omstaendlig commented 1 year ago

Some sites block requests made with the default user agent used in Python's requests module.

Hm.com, for instance, responds with a 403. So, even though the file is to be found at hm.com/security.txt, an exception is raised.

Rafiot commented 1 year ago

This is such a dumb idea from these websites... But noted, I'll add a feature to change the UA.

Note to self (or to whoever wants to implement it first), do it as it is implemented there: https://github.com/Lookyloo/PyLookyloo/blob/main/pylookyloo/api.py#L59