search

israel-dryer / Indeed-Job-Scraper

A web scraper to extract job postings from www.indeed.com
88 stars 57 forks source link

https status code 403 Forbidden #4

Open Blurdroid opened 1 year ago

Blurdroid commented 1 year ago

is any one have the same problem when i fetch and extract the raw HTML ?

nidhinnaushad commented 1 year ago

A status code 403 is when the user side has any issues, verify the connectivity, and also the id and class name that your trying to filter out. There are changes to the Indeed web page according to the region of access.

malizd commented 1 year ago

Same here. The URL when browsed on web browser works fine, but not from the Python script. The issue is at:

response = requests.get(url)

The response is landing page in Cloudflare indicating the website is protected by Cloudflare for security issues triggered [by script]. (https://www.cloudflare.com/en-gb/5xx-error-landing/)

Would there be any way around it?