Current master is failing build test on Ubuntu 18.0.4.5

[dgtlmoon]

Python3.6 on ubuntu here

git clone https://github.com/LeoHsiao1/pyexiv2.git
cd pyexiv2

cd pyexiv2/lib/
g++ exiv2api.cpp -o py36-linux/exiv2api.so -O3 -Wall -std=c++11 -shared -fPIC `python3 -m pybind11 --includes` -I /usr/include/exiv2 -I /usr/include/python3 -L ./lib -l exiv2
cd ../..
python3 setup.py install
pytest -v

pyexiv2/tests/test_func.py::test_read_exif PASSED                                                                                                                                                          [  3%]
pyexiv2/tests/test_func.py::test_read_iptc PASSED                                                                                                                                                          [  7%]
pyexiv2/tests/test_func.py::test_read_xmp PASSED                                                                                                                                                           [ 11%]
pyexiv2/tests/test_func.py::test_read_raw_xmp FAILED                                                                                                                                                       [ 15%]
pyexiv2/tests/test_func.py::test_modify_exif PASSED                                                                                                                                                        [ 19%]
pyexiv2/tests/test_func.py::test_modify_iptc PASSED                                                                                                                                                        [ 23%]
pyexiv2/tests/test_func.py::test_modify_xmp PASSED                                                                                                                                                         [ 26%]
pyexiv2/tests/test_func.py::test_clear_exif PASSED                                                                                                                                                         [ 30%]
pyexiv2/tests/test_func.py::test_clear_iptc PASSED                                                                                                                                                         [ 34%]
pyexiv2/tests/test_func.py::test_clear_xmp PASSED                                                                                                                                                          [ 38%]
pyexiv2/tests/test_func.py::test_nonexistent_path PASSED                                                                                                                                                   [ 42%]
pyexiv2/tests/test_func.py::test_not_image_path PASSED                                                                                                                                                     [ 46%]
pyexiv2/tests/test_func.py::test_error_log PASSED                                                                                                                                                          [ 50%]
pyexiv2/tests/test_func_on_ImageData.py::test_read_all FAILED                                                                                                                                              [ 53%]
pyexiv2/tests/test_func_on_ImageData.py::test_modify_exif PASSED                                                                                                                                           [ 57%]
pyexiv2/tests/test_func_on_ImageData.py::test_modify_iptc PASSED                                                                                                                                           [ 61%]
pyexiv2/tests/test_func_on_ImageData.py::test_modify_xmp PASSED                                                                                                                                            [ 65%]
pyexiv2/tests/test_func_on_ImageData.py::test_clear_all PASSED                                                                                                                                             [ 69%]
pyexiv2/tests/test_func_on_ImageData.py::test_error_log PASSED                                                                                                                                             [ 73%]
pyexiv2/tests/test_perf.py::test_memory_leak_when_reading FAILED                                                                                                                                           [ 76%]
pyexiv2/tests/test_perf.py::test_memory_leak_when_writing PASSED                                                                                                                                           [ 80%]
pyexiv2/tests/test_perf.py::test_stack_overflow PASSED                                                                                                                                                     [ 84%]
pyexiv2/tests/test_perf.py::test_transmit_various_characters PASSED                                                                                                                                        [ 88%]
pyexiv2/tests/test_perf_on_ImageData.py::test_memory_leak_when_reading FAILED                                                                                                                              [ 92%]
pyexiv2/tests/test_perf_on_ImageData.py::test_memory_leak_when_writing PASSED                                                                                                                              [ 96%]
pyexiv2/tests/test_perf_on_ImageData.py::test_stack_overflow PASSED                                                                                                                                        [100%]

==================================================================================================== FAILURES ====================================================================================================
_______________________________________________________________________________________________ test_read_raw_xmp ________________________________________________________________________________________________

    @check_md5
    def test_read_raw_xmp():
        with Image(path) as img:
>           assert len(img.read_raw_xmp()) == 4593
E           assert 4635 == 4593
E             +4635
E             -4593

pyexiv2/tests/test_func.py:28: AssertionError
_________________________________________________________________________________________________ test_read_all __________________________________________________________________________________________________

    @check_md5
    def test_read_all():
        with open(path, 'rb') as f:
            with ImageData(f.read()) as img:
                compare_dict(testdata.EXIF, img.read_exif())
                compare_dict(testdata.IPTC, img.read_iptc())
                compare_dict(testdata.XMP, img.read_xmp())
>               assert len(img.read_raw_xmp()) == 4593
E               assert 4635 == 4593
E                 +4635
E                 -4593

pyexiv2/tests/test_func_on_ImageData.py:12: AssertionError
_________________________________________________________________________________________ test_memory_leak_when_reading __________________________________________________________________________________________

    @check_md5
    def test_memory_leak_when_reading():
        p = psutil.Process(os.getpid())
        m0 = p.memory_info().rss
        for _ in range(1000):
            test_func.test_read_exif()
            test_func.test_read_iptc()
            test_func.test_read_xmp()
>           test_func.test_read_raw_xmp()

pyexiv2/tests/test_perf.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyexiv2/tests/base.py:37: in wrapper
    ret = func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @check_md5
    def test_read_raw_xmp():
        with Image(path) as img:
>           assert len(img.read_raw_xmp()) == 4593
E           assert 4635 == 4593
E             +4635
E             -4593

pyexiv2/tests/test_func.py:28: AssertionError
_________________________________________________________________________________________ test_memory_leak_when_reading __________________________________________________________________________________________

    @check_md5
    def test_memory_leak_when_reading():
        p = psutil.Process(os.getpid())
        m0 = p.memory_info().rss
        for _ in range(1000):
>           test_func_on_ImageData.test_read_all()

pyexiv2/tests/test_perf_on_ImageData.py:13: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pyexiv2/tests/base.py:37: in wrapper
    ret = func(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    @check_md5
    def test_read_all():
        with open(path, 'rb') as f:
            with ImageData(f.read()) as img:
                compare_dict(testdata.EXIF, img.read_exif())
                compare_dict(testdata.IPTC, img.read_iptc())
                compare_dict(testdata.XMP, img.read_xmp())
>               assert len(img.read_raw_xmp()) == 4593
E               assert 4635 == 4593
E                 +4635
E                 -4593

pyexiv2/tests/test_func_on_ImageData.py:12: AssertionError
============================================================================================ short test summary info =============================================================================================
FAILED pyexiv2/tests/test_func.py::test_read_raw_xmp - assert 4635 == 4593
FAILED pyexiv2/tests/test_func_on_ImageData.py::test_read_all - assert 4635 == 4593
FAILED pyexiv2/tests/test_perf.py::test_memory_leak_when_reading - assert 4635 == 4593
FAILED pyexiv2/tests/test_perf_on_ImageData.py::test_memory_leak_when_reading - assert 4635 == 4593
========================================================================================= 4 failed, 22 passed in 10.42s ==========================================================================================
d````

[LeoHsiao1]

I've included whitespace in reference_data.py without ignoring any characters.

Finally, you see the key problem. Exiv2 returns x:xmptk="XMP Core 4.4.0-Exiv2". I haven't investigated the cause of it. https://github.com/LeoHsiao1/pyexiv2/issues/41#issuecomment-753575807

[dgtlmoon]

OK - do you agree to this

@check_md5
def test_read_raw_xmp():
    with Image(test_img) as img:
        f = open("test_read_raw_xmp.txt", "w+")
        f.write(img.read_raw_xmp())
        diff_text(reference_data.RAW_XMP, img.read_raw_xmp())

I see in test_read_raw_xmp.txt

Ubuntu 18.0.4.5 (me) - x:xmptk="Adobe XMP Core 5.6-c128 79.159124, 2016/03/18-14:01:55 "> (same as in image) Ubuntu 18.0.4.2 (you) - x:xmptk="XMP Core 4.4.0-Exiv2 "> (something else)

Can you confirm yes/no that you see this in your test_read_raw_xmp.txt ?

[LeoHsiao1]

Yeah, that's what I got with exiv2 : x:xmptk="XMP Core 4.4.0-Exiv2" So I use it as a reference output for test cases.

[dgtlmoon]

@LeoHsiao1 OK, maybe nice idea is to include the output of img.read_raw_xmp() in the debug test verbose output, so then we can see how it looks in github actions too..

[dgtlmoon]

Something in your setup is weird, if what you're saying is true..

The current version of Exiv2 I am using is 0.27.2. Ubuntu 18.0.4.2 (you)

because the current 18.0.4 LTS package should be 0.25-3

https://ubuntu.pkgs.org/18.04/ubuntu-main-amd64/libexiv2-14_0.25-3.1_amd64.deb.html

are you running something weird like testing ?

[dgtlmoon]

How do you have 0.27 ?

[LeoHsiao1]

Well. According to the document pyexiv2/lib, you should download the distribution of Exiv2 from the website.

Download the release version of Exiv2, unpack it.
  - Linux64 : <https://www.exiv2.org/archive.html>

[dgtlmoon]

This project gets more insane,

Your perspective "We tested for ubuntu 18.04 but using a version of the library that is not shipped lib 18.04"

fuck man :/ sorry, this is just completely insane

[dgtlmoon]

I cant contribute here anymore, it's just too insane

[dgtlmoon]

Is there a reason why you insist on a version of a library that is not shipped with that distribution? I bet you did not bother to document that reason?

[LeoHsiao1]

Of course, I am using the latest version as possible.

• Version 0.25.3 of Exiv2 was released in 2016.
• Version 0.27.2 of Exiv2 was released in 2019.
• Version 0.27.3 of Exiv2 was released in 2020.

So your thoughts are strange: Why did you stick with the old version 4 years ago?

This statement is false: a version of a library that is not shipped with that distribution Pkgs.org does not include Exiv2 0.27 because no one updated the release package above, not because Exiv2 0.27 does not support Ubuntu 18.04.

Anyway, when you download a software, you always look to see if the package exists on Pkgs.org, rather than the official website of the software? That idea is crazy! Pkgs.org doesn't update packages very often, they are just volunteers.

If you are a beginner, I suggest you download general software in these ways:

• Download the compiled package from the software's official website or from the Ubuntu Archive
• Download the source package from the software's official website and compile it yourself.

[LeoHsiao1]

By the way, the CI process of Exiv2 is running on ubuntu18.0.4. If you build from the source package, you can execute Exiv2 on a lower version of Ubuntu.

[dgtlmoon]

USE THE ACTUAL LIBRARY THAT IS SHIPPED WITH THE DISTRIBUTION

THIS STOPS YOU FROM HAVING TO MANUALLY UPGRADE THE LIBRARY WHEN THERE IS A SECURITY ISSUE FOUND!

THIS IS THE WHOLE REASON FOR USING THE ACTUAL LIBRARY THAT IS SHIPPED WITH THE DISTRIBUTION !!

THERE IS NOTHING WRONG WITH USING AN "OLDER" VERSION IF IT STILL FUNCTIONS FINE!

it's all about managing the risks, you cannot expect the site owner to check his exiv version for updates/security issues every night

this is insane!

[LeoHsiao1]

1. In your opinion, the older the software, the more secure it is. So why are you using ubuntu18.04, not Ubuntu 14.04, Ubuntu 4.04 ?

2. In your opinion, ubuntu18.04 is not safe to use exiv2 0.27.2. However, exiv2 does not officially say this and has been using ubuntu18.04 as the CI environment. Your opinion is not reasonable.

[LeoHsiao1]

To make matters worse: Exiv2 fixes a number of problems with each version update: https://www.exiv2.org/changelog.html If you use an older version, you can be attacked with a known defect. Surprise!

[LeoHsiao1]

I just released v2.4.1 , which adds APIs for handling Comment embedded in images. Although the API was implemented two weeks ago, I spent the rest of the time adding APIs for handling ICC, improving test cases, and improving the Workflow scripts.

[github-actions[bot]]

This issue has been automatically closed because there has been no activity for a month.