openzim / kolibri

Convert a Kolibri channel in ZIM file(s)
GNU General Public License v3.0
8 stars 12 forks source link

Enhance situation around videos encoding and caching #86

Closed benoit74 closed 7 months ago

benoit74 commented 7 months ago

Rationale

Fix #82

83 is not fully fixed but the situation is way better (we start only one ffmpeg process per scraper run), the rest will be done by https://github.com/openzim/python-scraperlib/issues/123

Changes

Tests

This has been tested with:

kolibri2zim --name "test" --output output --tmp-dir tmp --channel-id 878ec2e6f88c5c268b1be6f202833cd4 --node-ids '455c985cf13c59018d9ec1b27b25fdf3,c938b19c971d5c539d4367c07d4ceb8b,759b3f9a20eb5b5d96032faa948f3729,63a274c6174f50999f7011b5e8984e2b,0b64184b660a51faa2acc1977e937d2d,878ec2e6f88c5c268b1be6f202833cd4' --debug --low-quality --use-webm --optimization-cache $OPTIM_CACHE

First call re-encodes and uploads the video, second call uses S3 file.

codecov[bot] commented 7 months ago

Codecov Report

Attention: 31 lines in your changes are missing coverage. Please review.

Comparison is base (cb214f4) 29.75% compared to head (04fcdff) 29.42%. Report is 3 commits behind head on main.

Files Patch % Lines
src/kolibri2zim/scraper.py 25.00% 27 Missing :warning:
src/kolibri2zim/entrypoint.py 25.00% 3 Missing :warning:
src/kolibri2zim/processing.py 0.00% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #86 +/- ## ========================================== - Coverage 29.75% 29.42% -0.33% ========================================== Files 6 6 Lines 773 751 -22 Branches 135 134 -1 ========================================== - Hits 230 221 -9 + Misses 531 522 -9 + Partials 12 8 -4 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.