search

meilisearch / scrapix

MIT License
21 stars 9 forks source link

fix(crawler): fix `failed` webhook #80

Open brunoocasali opened 1 year ago

brunoocasali commented 1 year ago

Today, when any error happens in the crawler the listener (webhook) is not being triggered.

This should be fixed, and the failed handler should be implemented in the listener as well (cloud)

bidoubiwa commented 1 year ago

I think it does trigger a failure. Can you provide an example?

brunoocasali commented 1 year ago

It comes from that perspective when we are hit by the cache. The user will see a timeout instead, but in reality, we have no data and no timeout error at all.

So even being a "expected" behavior, for the user perspective it is an error/failure.