PaulMcInnis / JobFunnel

Scrape job websites into a single spreadsheet with no duplicates.
MIT License
1.78k stars 210 forks source link

Exception is caught when calling scrape now; this should fix our CI Problems #130

Closed thebigG closed 3 years ago

thebigG commented 3 years ago

Hello everyone,

hope you are all doing well.

Description

When calling scrape function on scrapers, exceptions are caught and an error message is logged. This should fix our CI problems; JobFunnel is not just going to crash if it fails on one of the scrapes because of a CAPTCHA or something like that.

Context of change

Please add options that are relevant and mark any boxes that apply.

Type of change

Please mark any boxes that apply.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.

Checklist:

Please mark any boxes that have been completed.

thebigG commented 3 years ago

It failed because of #123. Don't merge PR now. Will look at it and fix it.

thebigG commented 3 years ago

edit: keep us in the loop on 123 i guess, but we may need to create a separate pr for fixing that issue

:+1:. You're right; it will make things simpler. I'll fix #123 and submit another PR, hopefully sometime this week.