orangecoding / fredy

:heart: Fredy - [F]ind [R]eal [E]states [D]amn Eas[y] - Fredy will constantly search for new listings on sites like Immoscout or Immowelt and send new results to you, so that you can focus on more important things in life ;)
http://www.orange-coding.net
MIT License
212 stars 54 forks source link

Make `requestDriver` more resilient to errors #46

Closed joschi closed 2 years ago

joschi commented 2 years ago

If the async request performed in requestDriver.makeDriver() fails, it would call the callback function with empty parameters but then continue the execution which can lead to the following error and crash of Fredy:

Error while trying to scrape data. Received error: Request failed with status code 504
/fredy/lib/services/requestDriver.js:25
    if (typeof result.data === 'object' && url.toLowerCase().indexOf('scrapingant') !== -1) {
                      ^

TypeError: Cannot read properties of undefined (reading 'data')
    at driver (/fredy/lib/services/requestDriver.js:25:23)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
orangecoding commented 2 years ago

@joschi a test is failing. I don't think this has something to do with your change, but can you have a look?

joschi commented 2 years ago

@orangecoding Looks like an intermittent error with wg-gesucht.de which has been fixed by now.

Successful test for wg-gesucht.de

Maybe you could re-run the tests on this PR.