[Windows] Crash when localised exception text has non-ASCII characters

[RSabet]

Hello,

first many thansk for providing the exiv2 library!

I am using exiv2==0.17.0, which as far as i understand should support JPEG XL

Python 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> exiv2.ImageFactory.open("precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl")

just crashes - no exception.

The image is just grabbed from here and renamed to test.jxl

Same code works fine when opening another file jxl file.

Any ideas?

reagrds ramin

[jim-easterbrook]

I can't reproduce the problem here.

$ python
Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> exiv2.__version__
'0.17.0'
>>> exiv2.versionInfo()
{'version': '0.28.3', 'EXV_ENABLE_NLS': True, 'EXV_ENABLE_BMFF': True, 'EXV_ENABLE_VIDEO': True, 'EXV_UNICODE_PATH': False, 'EXV_ENABLE_WEBREADY': True, 'EXV_USE_CURL': True, 'EXV_ENABLE_FILESYSTEM': True}
>>> im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
exiv2.Exiv2Error: (<ErrorCode.kerFileContainsUnknownImageType: 12>, 'precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl: The file contains data of an unknown image type')
>>> 

[jim-easterbrook]

Same on Windows 7:

Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (
AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> exiv2.versionInfo()
{'version': '0.28.3', 'EXV_ENABLE_NLS': True, 'EXV_ENABLE_BMFF': True, 'EXV_ENAB
LE_VIDEO': True, 'EXV_UNICODE_PATH': False, 'EXV_ENABLE_WEBREADY': True, 'EXV_US
E_CURL': True, 'EXV_ENABLE_FILESYSTEM': True}
>>> im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-wi
th-robotic-exactitude.jxl')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
exiv2.Exiv2Error: (<ErrorCode.kerFileContainsUnknownImageType: 12>, 'precision-m
achinery-shapes-golden-substance-with-robotic-exactitude.jxl: The file contains
data of an unknown image type')

[jim-easterbrook]

Same with Python 3.12 on Linux:

Python 3.12.3 (main, Jul 31 2024, 17:43:48) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> exiv2.__version__
'0.17.0'
>>> exiv2.versionInfo()
{'version': '0.28.3', 'EXV_ENABLE_NLS': True, 'EXV_ENABLE_BMFF': True, 'EXV_ENABLE_VIDEO': True, 'EXV_UNICODE_PATH': False, 'EXV_ENABLE_WEBREADY': True, 'EXV_USE_CURL': True, 'EXV_ENABLE_FILESYSTEM': True}
>>> im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
exiv2.Exiv2Error: (<ErrorCode.kerFileContainsUnknownImageType: 12>, 'precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl: The file contains data of an unknown image type')

I don't have a recent Windows version capable of running Python 3.12, so I can't reproduce your exact setup.

[jim-easterbrook]

Looking at the file data I see Webkit-logo-P3.jxl appears to start with an obvious JXL header.

0000000 nul nul nul  ff   J   X   L  sp  cr  nl bel  nl nul nul nul dc4
0000020   f   t   y   p   j   x   l  sp nul nul nul nul   j   x   l  sp
0000040 nul nul nul  ht   j   x   l   l  nl nul nul   )   6   j   x   l

precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl starts quite differently.

0000000 del  nl   :   !   h   o nul eot   b   B nul dc3  bs stx nul   @
0000020   l can  ht dle   s   Q   n dc3   "   U   b dc3 ack dc2  bs syn
0000040  si   Z   V   Z dle dc4   f dc4 can   R   #   Q   c dle   H soh

This doesn't explain your python-exiv2 crashes on opening though.

[RSabet]

Thanks for the fast response. I also tested on linux, works fine, on Windows 11 23H2 tried two different python versions 3.11.5 and 3.8.10 you tried, both crashed :-(

Python 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import exiv2 im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl') Traceback (most recent call last): File "", line 1, in exiv2.Exiv2Error: (<ErrorCode.kerFileContainsUnknownImageType: 12>, 'precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl: The file contains data of an unknown image type')

Windows:

Python 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:38:34) [MSC v.1936 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.

import exiv2 im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl') --- crash ---

Python 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.

import exiv2 exiv2.versionInfo() {'version': '0.28.3', 'EXV_ENABLE_NLS': True, 'EXV_ENABLE_BMFF': True, 'EXV_ENABLE_VIDEO': True, 'EXV_UNICODE_PATH': False, 'EXV_ENABLE_WEBREADY': True, 'EXV_USE_CURL': True, 'EXV_ENABLE_FILESYSTEM': True} im = exiv2.ImageFactory.open('precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl') --- crash ---

[jim-easterbrook]

Have you tried the exiv2 command line tool? You can download a Windows build from https://exiv2.org/download.html and try it with your problem file: exiv2 -pa precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl

Or maybe python-exiv2 crashes on any libexiv2 failure:

Python 3.6.15 (default, Sep 23 2021, 15:41:43) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> key = exiv2.ExifKey('not.valid.tag')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
exiv2.Exiv2Error: (<ErrorCode.kerInvalidKey: 7>, "Invalid key 'not.valid.tag'")

[RSabet]

good idea, but i get the same output as on linux ...

exiv2.exe -pa .\precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl

Exiv2 exception in print action for file .\precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl:
.\precision-machinery-shapes-golden-substance-with-robotic-exactitude.jxl: The file contains data of an unknown image type

You were absolutely right with your second option:

Python 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> key = exiv2.ExifKey('not.valid.tag')

xxx crash xxx

[jim-easterbrook]

OK, it's a problem in the Windows python-exiv2 exception handling. Does your Windows machine have a non-English locale? If so, can you try deleting the python-exiv2 'locale' directory. ('pip show exiv2' will show the site-packages directory where it's installed, site-packages/exiv2/locale should be the locale directory.)

[jim-easterbrook]

Eureka! I've successfully reproduced the problem on my Windows 7 virtual machine running Python 3.8. I had to set a German locale to get it to crash.

I thought localisation on Windows was working OK as the relevant test passed in the Windows builds: https://github.com/jim-easterbrook/python-exiv2/actions/runs/10217599937/job/28271760154#step:11:1507

However, the translated string being tested doesn't have any non-ASCII characters! (Unlike "Ungültiger Schlüssel".)

[jim-easterbrook]

Commit 0c42f02 modifies the test so it now fails.

[jim-easterbrook]

I think I've found the problem (after trying a lot of things that aren't the problem). Because exiv2 v0.28 drops use of Windows wchar interfaces I'd got it into my head that it was using utf-8 for error messages. However, the localisation initialisation has this:

#ifdef EXV_HAVE_BIND_TEXTDOMAIN_CODESET
    bind_textdomain_codeset(EXV_PACKAGE_NAME, "UTF-8");
#endif

This means the error message localisation uses the system encoding unless EXV_HAVE_BIND_TEXTDOMAIN_CODESET is defined. There is no mention of EXV_HAVE_BIND_TEXTDOMAIN_CODESET anywhere but that bit of source code.

Using my Windows CP to utf-8 conversion on exiv2 v0.28 exceptions as well as v0.27 seems to solve the problem.

Can you download the windows-28-wheels "artifact" from here: https://github.com/jim-easterbrook/python-exiv2/actions/runs/10922252696 It's a zip file with binary wheels for every Windows Python version, you should be able to install one of them with pip install wheel_file_name and test it.

[RSabet]

Great Work! I can confirm the new wheel solved the problem:

Python 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import exiv2
>>> exiv2.versionInfo()
{'version': '0.28.3', 'EXV_ENABLE_NLS': True, 'EXV_ENABLE_BMFF': True, 'EXV_ENABLE_VIDEO': True, 'EXV_UNICODE_PATH': False, 'EXV_ENABLE_WEBREADY': True, 'EXV_USE_CURL': True, 'EXV_ENABLE_FILESYSTEM': True}
>>> key = exiv2.ExifKey('not.valid.tag')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
exiv2.Exiv2Error: (<ErrorCode.kerInvalidKey: 7>, "Ungültiger Schlüssel 'not.valid.tag'")

[jim-easterbrook]

Thanks. I'll delete that action run now that you've tested it and get a proper new release out in a day or two.

[jim-easterbrook]

Now released as version 0.17.1 Thank you for reporting the problem and helping me improve python-exiv2.