GUI / covid-vaccine-spotter

https://www.vaccinespotter.org
MIT License
508 stars 136 forks source link

Website blocked #120

Closed H4CKL0RD closed 3 years ago

H4CKL0RD commented 3 years ago

The website is forbidden error 403, if I cannot use this website to crawl, which other ones can I use that have multiple locations?

GUI commented 3 years ago

@H4CKL0RD: Can you provide more details? Where are you seeing 403 forbidden errors? On the web page in a browser, or are you trying to crawl with some other tools? I'm not aware of anything that would lead to 403 errors, but it's possible CloudFlare might be doing something. If you're looking to crawl the site, have you seen the APIs?

H4CKL0RD commented 3 years ago

Ok I haven't seen the api's since this is mostly what i was trying to do (its in python, sharing my code here)

`from bs4 import BeautifulSoup as soup from urllib.request import urlopen as uReq state = input("What state are you in(reply with state abv): ") zipcode = input("What is your zip code?: ") VaccineType = input("Do you want to choose a type of vaccine? If so tell the type of vaccine(Johnsen+Johnsen(just type jj), moderna, and pfizer)if not type none: ")

if VaccineType == "jj": url =f"https://www.vaccinespotter.org/{state}/?zip={zipcode}&vaccine_type=jj&radius=100" if VaccineType == "pfizer": url = f"https://www.vaccinespotter.org/{state}/?zip={zipcode}&vaccine_type=pfizer&radius=100" if VaccineType == "moderna": url = f"https://www.vaccinespotter.org/{state}/?zip={zipcode}&vaccine_type=moderna&radius=100" else: url = f"https://www.vaccinespotter.org/{state}X/?zip={zipcode}&radius=100&include_all=true"

print (url)

uClient = uReq(url) page_html = uClient.read(url)

uClient.close()

page_soup = soup(page_html, "html.parser")

page_soup.h1`

H4CKL0RD commented 3 years ago

I just had a test for finding the header and i got this error raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 403: Forbidden

H4CKL0RD commented 3 years ago

hello?? @GUI

H4CKL0RD commented 3 years ago

Nevermind, i was using urllib instead of requests. My bad.