search

sefinek / Sefinek-Blocklist-Collection

🌍 A comprehensive repository of blocklists for various DNS servers, featuring over 100 links and more than 6 million domains. Additionally, you can use our personalized Blocklist Generator to tailor content filtering according to your specific needs, giving you full control over what gets blocked on your network.
https://blocklist.sefinek.net
Other
554 stars 26 forks source link

Server responds with Forbidden 403 Error for Python users #30

Closed hl2guide closed 5 months ago

hl2guide commented 5 months ago

e.g.:

FILENAME = CURRENTWORKINGDIRECTORY + "downloaded_lists\\blocklist" + str(LIST_INDEX) + ".txt"
URL = "https://blocklist.sefinek.net/generated/v1/adguard/abuse/blocklistproject/hosts.fork.txt"
urlretrieve(URL, FILENAME)
sefinek commented 5 months ago

I don't see any logs indicating blocked requests. Could you please provide the RayID?

sefinek commented 5 months ago

I have just found it now:

Ray ID: 88b3b16c0882aadd
User agent: Python-urllib/3.12
Browser integrity check

Try changing the user agent.

hl2guide commented 5 months ago

Thanks, got it working with:

import requests
url = "https://blocklist.sefinek.net/generated/v1/adguard/abuse/blocklistproject/hosts.fork.txt"
headers = {
    'User-Agent': 'Mozilla 5.0',
}
response = requests.get(url, headers=headers)
content = response.text

if response.status_code == 200:
    with open("output.txt", "w") as file:
        file.write(content)
sefinek commented 5 months ago

Great, no problem <:

Anyway, I recommend using user agents like these:

NAME/VERSION (+HOMEPAGE) or Mozilla/5.0 (compatible; NAME/VERSION; +HOMEPAGE)