Such errors as page.waitForSelector() timeout inside the "Extend Output Function" won't be logged on regular (not debug) log level

cermak-petr / actor-zillow-api-scraper

Apify actor for extracting data about homes from Zillow.com using it's internal API.

https://apify.com/petr_cermak/zillow-api-scraper

Apache License 2.0

49 stars 42 forks source link

Such errors as page.waitForSelector() timeout inside the "Extend Output Function" won't be logged on regular (not debug) log level #55

Closed vladyslav-n closed 2 years ago

vladyslav-n commented 2 years ago

https://github.com/cermak-petr/actor-zillow-api-scraper/blob/fc46b9e8253e2da7865f7ba9a02f7c43b9cfc283/src/page-handler.js#L243

I guess a lot of errors cannot be caught by that RegExp. An example is TimeoutError thrown by page.waitForSelector() inside the "Extend Output Function", and you may see that error log only by setting the log level to DEBUG, which is too verbose to be used just to see an error inside the provided "Extend Output Function".

pocesar commented 2 years ago

you're right, but the extend scraper function is indeed a dev-focused feature, that requires you enabling the debugLog when using it 😄 but I'm forwarding the error to log.error now