Test error after upgrading to python-libzim3.1

FledgeXu commented 1 year ago

system:

Darwin Fledges-MBP 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar  6 20:59:58 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T6020 arm64

logs:

```tox .pkg: _optional_hooks> python /Users/fledge/Developer/python-scraperlib/.env/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ .pkg: get_requires_for_build_sdist> python /Users/fledge/Developer/python-scraperlib/.env/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ .pkg: prepare_metadata_for_build_wheel> python /Users/fledge/Developer/python-scraperlib/.env/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ .pkg: build_sdist> python /Users/fledge/Developer/python-scraperlib/.env/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ py311: install_package> python -I -m pip install --force-reinstall --no-deps /Users/fledge/Developer/python-scraperlib/.tox/.tmp/package/15/zimscraperlib-3.0.0.tar.gz py311: commands[0]> pytest --runslow --runinstalled --cov=zimscraperlib --cov-report=term --cov-report term-missing ============================= test session starts ============================== platform darwin -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0 cachedir: .tox/py311/.pytest_cache rootdir: /Users/fledge/Developer/python-scraperlib plugins: cov-4.0.0 collected 319 items tests/download/test_download.py ................. [ 5%] tests/filesystem/test_filesystem.py ..... [ 6%] tests/html/test_html.py ...... [ 8%] tests/i18n/test_i18n.py ......................... [ 16%] tests/image/test_image.py .............................................. [ 31%] .......................... [ 39%] tests/inputs/test_inputs.py .......... [ 42%] tests/logging/test_logging.py ............... [ 47%] tests/misc/test_misc.py .... [ 48%] tests/ogvjs/test_ogvjs.py ... [ 49%] tests/types/test_types.py .............. [ 53%] tests/uri/test_uri.py .............. [ 57%] tests/video/test_video.py ....F.................. [ 65%] tests/zim/test_archive.py ......... [ 68%] tests/zim/test_fs.py ....... [ 70%] tests/zim/test_libkiwix.py ................ [ 75%] tests/zim/test_zim_creator.py F......................................... [ 88%] ..................................... [100%] =================================== FAILURES =================================== _________________ test_get_media_info[mp4-video.mp4-expected1] _________________ media_format = 'mp4', media = 'video.mp4' expected = {'bitrate': 3818365, 'codecs': ['h264', 'aac'], 'duration': 2} test_files = {'mkv': PosixPath('/Users/fledge/Developer/python-scraperlib/tests/files/video.mkv'), 'mp3': PosixPath('/Users/fledge/...aperlib/tests/files/video.mp4'), 'webm': PosixPath('/Users/fledge/Developer/python-scraperlib/tests/files/video.webm')} @pytest.mark.slow @pytest.mark.parametrize( "media_format,media,expected", [ ( "mkv", "video.mkv", {"codecs": ["h264", "vorbis"], "duration": 2, "bitrate": 3819022}, ), ( "mp4", "video.mp4", {"codecs": ["h264", "aac"], "duration": 2, "bitrate": 3818365}, ), ( "webm", "video.webm", {"codecs": ["vp9", "opus"], "duration": 2, "bitrate": 336650}, ), ( "mp3", "audio.mp3", {"codecs": ["mp3"], "duration": 2, "bitrate": 129066}, ), ], ) def test_get_media_info(media_format, media, expected, test_files): with tempfile.TemporaryDirectory() as t: src = pathlib.Path(t).joinpath(media) shutil.copy2(test_files[media_format], src) > assert get_media_info(src) == expected E AssertionError: assert {'bitrate': 3...'duration': 2} == {'bitrate': 3...'duration': 2} E Omitting 2 identical items, use -vv to show E Differing items: E {'bitrate': 3837275} != {'bitrate': 3818365} E Use -v to get more diff tests/video/test_video.py:134: AssertionError _______________________________ test_zim_creator _______________________________ tmp_path = PosixPath('/private/var/folders/2h/kc3zwff52ws96r__pgrrqp540000gn/T/pytest-of-fledge/pytest-6/test_zim_creator0') png_image = PosixPath('/Users/fledge/Developer/python-scraperlib/tests/files/commons48.png') html_file = PosixPath('/private/var/folders/2h/kc3zwff52ws96r__pgrrqp540000gn/T/pytest-of-fledge/pytest-6/test_zim_creator0/test.html') html_str = '\n\n

This looks like a bug in python-libzim, not python-scraperlib.

rgaudin commented 1 year ago

Thanks for the report ; there are two issue:

different bit-rate returned by get_media_info(). I can't reproduce. Definitely not tied to libzim. Since this is returned directly from ffprobe, I'm not much worried but we should find out what's happening:
- is it due to your arch?
- is it due to your ffprobe version? Can you share which version you use? Can you open a ticket?
missing suggestion results. This is due to a change in libzim indeed. I've changed the test in a7185fd

FledgeXu commented 1 year ago

I tested it with the ffprobe 5.1.2 and the bug is gone. It seems that ffmpeg 6.0 introduced this bug.

rgaudin commented 1 year ago

Even the duration is returned differently

❯ diff ffprobe5.1.2.csv ffprobe6.0.7.csv
3c3
< format,2.638000,3818365
---
> format,2.625000,3837275

openzim / python-scraperlib

Test error after upgrading to python-libzim3.1 #102