JennAckerman
commented
4 years ago Not sure if you had fixed the issue by now, but I figured I would tell how I fixed my StaleElementReferenceException just in case.
That exception is given because something in the web page had updated while the code was running, which changed the reference that the code was going to. In this instance, it's the job postings on the page.
For me, the job postings were not updating to the "Data Scientist" keyword until after the code closed the popup, making the code look for the old job postings that weren't there anymore.
In the 'glassdoor_scraper' I had to change the quotes code around the keyword in the url from:
"' + keyword + '"
to:
'+ keyword +'
Hope this helps!
The error comes from the line : driver.execute_script("arguments[0].click();", job_button) Note : I used this instead of your original code : #job_button.click() #You might
But now as I iterate over 500 or more , I am getting StaleElementReferenceException error,
Traceback (most recent call last): File "C:\Users\suy\Documents\Github Backup\Glassdoor_Salary_Project\data_collection.py", line 15, in
dataframe = sc.get_jobs("data scientist",frame,False,path,2)
File "C:\Users\suy\Documents\Github Backup\Glassdoor_Salary_Project\glassdoor_scrapper.py", line 63, in get_jobs driver.execute_script("arguments[0].click();", job_button)
File "C:\Users\suy.conda\envs\nlp\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 636, in execute_script 'args': converted_args})['value']
File "C:\Users\suy.conda\envs\nlp\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response)
File "C:\Users\suy.conda\envs\nlp\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace)
StaleElementReferenceException: stale element reference: element is not attached to the page document (Session info: chrome=83.0.4103.116)
Please let me know if you know this error.