Many fixes for reliability of the scraper

[benoit74]

Fix #74 Fix #76 Fix #77 (and glossary had the same problem)

Workaround for #71 (real solution postponed to "later") and many other likely situations where we encounter an "unknown" src/href/srcset (inline JS and CSS, ...)

Changes: see list of commits

Some remarks:

• with these changes, we tend to more often display only warning in the logs for situation which are known to not be totally OK but not cause significant harm to the final ZIM ; this is expected to be able to create a ZIM even when weird situation are encountered in the wild ; I tend to prefer these warnings to really handling the situation to be more easily able to investigate without debugging (e.g. for empty index or glossary pages, I preferred to log a warning and be able to investigate rather than just handling the case where it might be empty and saying nothing, this is not really normal and might be a real problem, we just don't know without manual analysis)
• I dropped the --html-issues-warn-only because now we do not have such issues anymore, we are always just logging a warning because these issues are here to stay and we do not want to fail the scrape just for few isolated issues (they are really isolated from test scrape which ran last days)
• as discussed, we know have a new dev option to control how many assets are allowed to fail (to provide some slack when needed) and a prod option to register known bad assets (and we automatically include some known ones coming from automatic CSS)

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 13.23529% with 118 lines in your changes missing coverage. Please review.

Project coverage is 43.14%. Comparing base (5236501) to head (ad99067). Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
scraper/src/mindtouch2zim/processor.py	1.04%	95 Missing :warning:
scraper/src/mindtouch2zim/asset.py	23.07%	20 Missing :warning:
scraper/src/mindtouch2zim/entrypoint.py	0.00%	1 Missing :warning:
scraper/src/mindtouch2zim/html_rewriting.py	88.88%	1 Missing :warning:
scraper/src/mindtouch2zim/utils.py	0.00%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #78 +/- ## ========================================== - Coverage 43.85% 43.14% -0.72% ========================================== Files 15 15 Lines 969 978 +9 Branches 133 133 ========================================== - Hits 425 422 -3 - Misses 529 545 +16 + Partials 15 11 -4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[benoit74]

For the record, I abused the dev Docker image by building from this branch as well, just to be able to run the scraper asap in Zimfarm, since we did not release 0.1, who cares

[benoit74]

I've opened https://github.com/openzim/mindtouch/issues/91 for the Exception, there are many more than the ones you're mentioning here. But good point indeed