stefanpf / jobsiteScrape.py

Scrapes a number of German job sites and dumps the scraped job offers into a XLSX spreadsheet.
5 stars 7 forks source link

jobsiteScrape.py -> scrape_stepstone() -> soup.select in line 115 returns empty list #8

Open stefanpf opened 5 years ago

stefanpf commented 5 years ago

Problem in line 115:

job_desc = soup.select('.offer__content')[1].getText()

Expected behaviour:

Return inner text for second page element with class .offer__content.

Bug:

Crashes with:

Traceback (most recent call last):
  File "jobsiteScrape.py", line 251, in <module>
    scrape_stepsone(stepstone_search_string)
  File "jobsiteScrape.py", line 115, in scrape_stepstone
    job_desc = soup.select('.offer__content')[1].getText()
IndexError: list index out of range

Tried:

Update 2018-11-08:

This is very weird. Two test runs this morning didn't show any error.

stefanpf commented 5 years ago

Quick fix: wrapped line 115 in try/except clause to prevent script from crashing.