search

Exiv2 / exiv2

Image metadata library and tools
http://www.exiv2.org/
Other
917 stars 278 forks source link

Build exiv2 on windows/msys2 #1449

Closed heckflosse closed 3 years ago

heckflosse commented 3 years ago

I want to contribute to exiv2 a bit and today I tried to build it on an up to date windows/msys2, but failed.

I could solve at least one issue, but currently I'm stuck at a linker problem:

C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: CMakeFiles/xmpdump.dir/objects.a(xmpdump.cpp.obj):xmpdump.cpp:(.text$_ZN5Exiv210BasicErrorIcEC1INSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS_9ErrorCodeERKT_[_ZN5Exiv210BasicErrorIcEC1INSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEENS_9ErrorCodeERKT_]+0x97): undefined reference to `__imp__ZN5Exiv210BasicErrorIcE6setMsgEv'
collect2.exe: error: ld returned 1 exit status

I will try to solve this issue...

clanmills commented 3 years ago

Are you building the 0.27-maintenance branch?

We branched the code when we shipped 0.27 in December 2018. I'm not too sure of the status of 'master'. This looks like it's related the a template issue with the Error class in 'master'. I've never seen that issue on '0.27-maintenance'.

heckflosse commented 3 years ago

I tried to build the master branch. Yes, it's related to the Error class. I will try the 0.27-maintenance branch now.

heckflosse commented 3 years ago

Building 0.27-maintenance branch worked fine, but make install didn't work because of

Install the project...
-- Install configuration: ""
CMake Error at cmake_install.cmake:41 (file):
  file cannot create directory: C:/Program Files (x86)/exiv2/lib/pkgconfig.
  Maybe need administrative privileges.

My shell has admin privs, but for a 64bit app C:/Program Files (x86) is not the correct location. The error sounds familiar to me (saw it on one of the other projects I contributed). Have to think about it...

heckflosse commented 3 years ago

Adding -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" to the cmake command fixed the above issue

clanmills commented 3 years ago

Ah, that's a good AND a surprise:

  1. Good. 0.27-maintenance is building.
  2. Surprise that you're anywhere near c:/Program Files.

Suggestions:

  1. Look at the notes in README.md concerning MinGW/msys2
  2. Look at the appveyor script: appveyor_mingw_cygwin.yml

And Thank You. I really appreciate you getting involved.

heckflosse commented 3 years ago

The above mentioned -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" most likely is only needed for windows/msys2 builds. At least for librtprocess (where I needed to use it for windows/msys2 builds) it was not needed for linux builds iirc

clanmills commented 3 years ago

I strongly encourage you to use my script msys64.cmd to configure the environment. And install the dependencies and build chain as documented in README.md. If you don't do that, the toolchain/path on Windows is so vaguely defined, almost anything can happen.

You'll see in appveyor_mingw_cygwin.yml that I set up the environment on the CI based on the information in README.md

Of course, there's more than one way to skin a cat. However, this is my way to keep sane. (The occasional beer also helps).
lagunitas-ipa 36_1_1

heckflosse commented 3 years ago

@clanmills

I strongly encourage you to use my script msys64.cmd to configure the environment. And install the dependencies and build chain as documented in README.md. If you don't do that, the toolchain/path on Windows is so vaguely defined, almost anything can happen.

You'll see in appveyor_mingw_cygwin.yml that I set up the environment on the CI based on the information in README.md

If you don't mind, I will try a simpler approach... Let's see if it works...

clanmills commented 3 years ago

I don't mind at all. I love simpler ways.

Luis set up the CI, however he didn't deal with Cygwin and MinGW and I did that in December. I asked for advice on the AppVeyor Forum and nobody helped. https://help.appveyor.com/discussions/questions/51961-building-on-mingw-64-and-cygwin64

I've got it work. It's messy and of course the Visual Studio builds are in a different file: appveyor.yml

If you have ideas about simplifying this, I'll very happy to accept your help.

heckflosse commented 3 years ago

Just trying a windows/msys2 clang build...

heckflosse commented 3 years ago

clang build using

cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" -DCMAKE_BUILD_TYPE="release" -DCMAKE_C_COMPILER="clang" -DCMAKE_CXX_COMPILER="clang++" ..

also works

heckflosse commented 3 years ago

with clang, only 

cmake -G "MSYS Makefiles" .........

works, but not

cmake -G "Unix Makefiles" .........
heckflosse commented 3 years ago

I will continue tomorrow...

The occasional beer also helps

I will enjoy a Talisker now...

clanmills commented 3 years ago

Enjoy, my friend. Thanks for getting involved.

kmilos commented 3 years ago

@heckflosse What's wrong with -G"MSYS Makefiles"? AFAIK it's a must when building for mingw, and certainly what almost all mingw package recipes including mingw-w64-exiv2 use (when they use Cmake of course).

Btw there is also possibility of Ninja as of late too, and some packages are switching over. Need to try for exiv2...

My workflow for building exiv2 0.27-maintenance on mingw:

  1. Use the MSYS2 MinGW 64-bit console from the Start menu (not the MSYS2 MSYS one)
  2. Run the relevant cmake line from the PKGBUILD recipe above

Mmmm, Talisker. Almost as good as Caol Ila.

clanmills commented 3 years ago

I built clang/Unix Makefiles/MinGW/msys2 without issue.

I think the essential thing is that the documented build in README.md works. If folks want to build in other ways, that's fine. However supporting an infinite build matrix isn't possible. For example, if somebody says "I can't build 32 bit static libraries on Solaris with Ninja and cmake 3.15.3 and clang 1.2.3, I will investigate - however, I'm unlikely to change the documentation.

There was a major change in the MinGW/msys2 build system requested in October. So, I'm willing to change it when there is a reason to do so. #1356

clanmills commented 3 years ago

BTW, @heckflosse meet @kmilos.

@kmilos has done some very good work in the last few months to bring the DNG, Tiff-EP and Exif support totally up to current standards. And he added more easyaccess API. And probably something else that I've forgotten.

@heckflosse is a major contributor to RawTherapee and he's here for an open-source vacation and to help out for v0.27.4.

It's good to have your both in the room. Thanks for being here.

heckflosse commented 3 years ago

I just run make tests on msys2. Here's the log. Most of the tests passed, but not all. At the end I had to CTRL-C, because it hung without cpu usage.

$ make tests
[ 14%] Built target exiv2lib_int
[ 24%] Built target exiv2-xmp
[ 48%] Built target exiv2lib
[ 50%] Built target xmpdump
[ 53%] Built target exiv2
[ 54%] Built target remotetest
[ 55%] Built target conntest
[ 56%] Built target geotag
[ 58%] Built target exiv2json
[ 60%] Built target path-test
[ 62%] Built target iptctest
[ 64%] Built target exifvalue
[ 65%] Built target iptceasy
[ 66%] Built target key-test
[ 68%] Built target tiff-test
[ 70%] Built target ini-test
[ 71%] Built target iptcprint
[ 72%] Built target iotest
[ 73%] Built target addmoddel
[ 76%] Built target metacopy
[ 77%] Built target xmpparser-test
[ 78%] Built target exifdata
[ 79%] Built target largeiptc-test
[ 80%] Built target werror-test
[ 82%] Built target exifcomment
[ 83%] Built target taglist
[ 84%] Built target easyaccess-test
[ 85%] Built target mrwthumb
[ 86%] Built target convert-test
[ 87%] Built target stringto-test
[ 88%] Built target mmap-test
[ 90%] Built target getopt-test
[ 92%] Built target prevtest
[ 93%] Built target exifdata-test
[ 94%] Built target xmpparse
[ 95%] Built target write-test
[ 96%] Built target write2-test
[ 97%] Built target xmpprint
[ 98%] Built target exifprint
[100%] Built target xmpsample
Scanning dependencies of target tests

---- Running bash_tests ----

Running addmoddel.sh ...
all testcases passed.
Running conversions.sh ...
all testcases passed.
Running crw-test.sh ...
all testcases passed.
Running exifdata-test.sh ...
all testcases passed.
Running exiv2-test.sh ...
600c600,624
< 20110626_213900.psd: Failed to open the file
---
> 20110626_213900.psd  File name       : 20110626_213900.psd
> 20110626_213900.psd  File size       : 52770 Bytes
> 20110626_213900.psd  MIME type       : image/x-photoshop
> 20110626_213900.psd  Image size      : 150 x 91
> 20110626_213900.psd  Thumbnail       : None
> 20110626_213900.psd  Camera make     :
> 20110626_213900.psd  Camera model    :
> 20110626_213900.psd  Image timestamp :
> 20110626_213900.psd  File number     :
> 20110626_213900.psd  Exposure time   :
> 20110626_213900.psd  Aperture        :
> 20110626_213900.psd  Exposure bias   :
> 20110626_213900.psd  Flash           :
> 20110626_213900.psd  Flash bias      :
> 20110626_213900.psd  Focal length    :
> 20110626_213900.psd  Subject distance:
> 20110626_213900.psd  ISO speed       :
> 20110626_213900.psd  Exposure mode   :
> 20110626_213900.psd  Metering mode   :
> 20110626_213900.psd  Macro mode      :
> 20110626_213900.psd  Image quality   :
> 20110626_213900.psd  White balance   :
> 20110626_213900.psd  Copyright       :
> 20110626_213900.psd  Exif comment    :
>
2144c2168,2188
< 20110626_213900.psd: Failed to open the file
---
> 20110626_213900.psd   Exif.Image.ImageWidth                        Short       1  150
> 20110626_213900.psd   Exif.Image.ImageLength                       Short       1  91
> 20110626_213900.psd   Exif.Image.BitsPerSample                     Short       3  8 8 8
> 20110626_213900.psd   Exif.Image.Orientation                       Short       1  top, left
> 20110626_213900.psd   Exif.Image.SamplesPerPixel                   Short       1  3
> 20110626_213900.psd   Exif.Image.XResolution                       Rational    1  72
> 20110626_213900.psd   Exif.Image.YResolution                       Rational    1  72
> 20110626_213900.psd   Exif.Image.ResolutionUnit                    Short       1  inch
> 20110626_213900.psd   Exif.Image.Software                          Ascii      30  Adobe Photoshop CS5 Macintosh
> 20110626_213900.psd   Exif.Image.DateTime                          Ascii      20  2011:06:26 21:39:00
> 20110626_213900.psd   Exif.Image.ExifTag                           Long        1  218
> 20110626_213900.psd   Exif.Photo.ExifVersion                       Undefined   4  2.21
> 20110626_213900.psd   Exif.Photo.ColorSpace                        Short       1  Uncalibrated
> 20110626_213900.psd   Exif.Photo.PixelXDimension                   Long        1  150
> 20110626_213900.psd   Exif.Photo.PixelYDimension                   Long        1  91
> 20110626_213900.psd   Exif.Thumbnail.Compression                   Short       1  JPEG (old-style)
> 20110626_213900.psd   Exif.Thumbnail.XResolution                   Rational    1  72
> 20110626_213900.psd   Exif.Thumbnail.YResolution                   Rational    1  72
> 20110626_213900.psd   Exif.Thumbnail.ResolutionUnit                Short       1  inch
> 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormat         Long        1  366
> 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  0
2147d2190
< 20110626_213900.psd: Failed to open the file
2182c2225,2227
< 20110626_213900.psd: Failed to open the file
---
> Writing Exif data from 20110626_213900.psd to ./20110626_213900.exv
> Writing IPTC data from 20110626_213900.psd to ./20110626_213900.exv
> Writing XMP data from 20110626_213900.psd to ./20110626_213900.exv
2217c2262
< 20110626_213900.psd: Failed to open the file
---
> 20110626_213900.psd: Exif data doesn't contain a thumbnail
2219d2263
< 20110626_213900.exv: Failed to open the file
2222c2266
< 1,1541c1,1541
---
> 1,1562c1,1562
3763a3808,3828
> < 20110626_213900.psd   Exif.Image.ImageWidth                        Short       1  150
> < 20110626_213900.psd   Exif.Image.ImageLength                       Short       1  91
> < 20110626_213900.psd   Exif.Image.BitsPerSample                     Short       3  8 8 8
> < 20110626_213900.psd   Exif.Image.Orientation                       Short       1  top, left
> < 20110626_213900.psd   Exif.Image.SamplesPerPixel                   Short       1  3
> < 20110626_213900.psd   Exif.Image.XResolution                       Rational    1  72
> < 20110626_213900.psd   Exif.Image.YResolution                       Rational    1  72
> < 20110626_213900.psd   Exif.Image.ResolutionUnit                    Short       1  inch
> < 20110626_213900.psd   Exif.Image.Software                          Ascii      30  Adobe Photoshop CS5 Macintosh
> < 20110626_213900.psd   Exif.Image.DateTime                          Ascii      20  2011:06:26 21:39:00
> < 20110626_213900.psd   Exif.Image.ExifTag                           Long        1  218
> < 20110626_213900.psd   Exif.Photo.ExifVersion                       Undefined   4  2.21
> < 20110626_213900.psd   Exif.Photo.ColorSpace                        Short       1  Uncalibrated
> < 20110626_213900.psd   Exif.Photo.PixelXDimension                   Long        1  150
> < 20110626_213900.psd   Exif.Photo.PixelYDimension                   Long        1  91
> < 20110626_213900.psd   Exif.Thumbnail.Compression                   Short       1  JPEG (old-style)
> < 20110626_213900.psd   Exif.Thumbnail.XResolution                   Rational    1  72
> < 20110626_213900.psd   Exif.Thumbnail.YResolution                   Rational    1  72
> < 20110626_213900.psd   Exif.Thumbnail.ResolutionUnit                Short       1  inch
> < 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormat         Long        1  366
> < 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  0
5305a5371,5391
> > 20110626_213900.exv   Exif.Image.ImageWidth                        Short       1  150
> > 20110626_213900.exv   Exif.Image.ImageLength                       Short       1  91
> > 20110626_213900.exv   Exif.Image.BitsPerSample                     Short       3  8 8 8
> > 20110626_213900.exv   Exif.Image.Orientation                       Short       1  top, left
> > 20110626_213900.exv   Exif.Image.SamplesPerPixel                   Short       1  3
> > 20110626_213900.exv   Exif.Image.XResolution                       Rational    1  72
> > 20110626_213900.exv   Exif.Image.YResolution                       Rational    1  72
> > 20110626_213900.exv   Exif.Image.ResolutionUnit                    Short       1  inch
> > 20110626_213900.exv   Exif.Image.Software                          Ascii      30  Adobe Photoshop CS5 Macintosh
> > 20110626_213900.exv   Exif.Image.DateTime                          Ascii      20  2011:06:26 21:39:00
> > 20110626_213900.exv   Exif.Image.ExifTag                           Long        1  218
> > 20110626_213900.exv   Exif.Photo.ExifVersion                       Undefined   4  2.21
> > 20110626_213900.exv   Exif.Photo.ColorSpace                        Short       1  Uncalibrated
> > 20110626_213900.exv   Exif.Photo.PixelXDimension                   Long        1  150
> > 20110626_213900.exv   Exif.Photo.PixelYDimension                   Long        1  91
> > 20110626_213900.exv   Exif.Thumbnail.Compression                   Short       1  JPEG (old-style)
> > 20110626_213900.exv   Exif.Thumbnail.XResolution                   Rational    1  72
> > 20110626_213900.exv   Exif.Thumbnail.YResolution                   Rational    1  72
> > 20110626_213900.exv   Exif.Thumbnail.ResolutionUnit                Short       1  inch
> > 20110626_213900.exv   Exif.Thumbnail.JPEGInterchangeFormat         Long        1  366
> > 20110626_213900.exv   Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  0
5340c5426
< 20110626_213900.psd: Failed to open the file
---
> Erasing thumbnail data
5372c5458
< 20110626_213900.psd: Failed to open the file
---
> 20110626_213900.psd: Image does not contain an Exif thumbnail
5406c5492,5494
< 20110626_213900.psd: Failed to open the file
---
> Erasing Exif data from the file
> Erasing IPTC data from the file
> Erasing XMP data from the file
5498c5586,5590
< 20110626_213900.psd: Failed to open the file
---
> 20110626_213900.psd  File name       : 20110626_213900.psd
> 20110626_213900.psd  File size       : 48170 Bytes
> 20110626_213900.psd  MIME type       : image/x-photoshop
> 20110626_213900.psd  Image size      : 150 x 91
> 20110626_213900.psd: No Exif data found in the file
5532c5624,5626
< 20110626_213900.psd: Failed to open the file
---
> Writing Exif data from ./20110626_213900.exv to 20110626_213900.psd
> Writing IPTC data from ./20110626_213900.exv to 20110626_213900.psd
> Writing XMP data from ./20110626_213900.exv to 20110626_213900.psd
5534d5627
< 20110626_213900.exv: Failed to open the file
5537c5630
< 1,1541c1,1541
---
> 1,1562c1,1562
7078a7172,7192
> < 20110626_213900.psd   Exif.Image.ImageWidth                        Short       1  150
> < 20110626_213900.psd   Exif.Image.ImageLength                       Short       1  91
> < 20110626_213900.psd   Exif.Image.BitsPerSample                     Short       3  8 8 8
> < 20110626_213900.psd   Exif.Image.Orientation                       Short       1  top, left
> < 20110626_213900.psd   Exif.Image.SamplesPerPixel                   Short       1  3
> < 20110626_213900.psd   Exif.Image.XResolution                       Rational    1  72
> < 20110626_213900.psd   Exif.Image.YResolution                       Rational    1  72
> < 20110626_213900.psd   Exif.Image.ResolutionUnit                    Short       1  inch
> < 20110626_213900.psd   Exif.Image.Software                          Ascii      30  Adobe Photoshop CS5 Macintosh
> < 20110626_213900.psd   Exif.Image.DateTime                          Ascii      20  2011:06:26 21:39:00
> < 20110626_213900.psd   Exif.Image.ExifTag                           Long        1  218
> < 20110626_213900.psd   Exif.Photo.ExifVersion                       Undefined   4  2.21
> < 20110626_213900.psd   Exif.Photo.ColorSpace                        Short       1  Uncalibrated
> < 20110626_213900.psd   Exif.Photo.PixelXDimension                   Long        1  150
> < 20110626_213900.psd   Exif.Photo.PixelYDimension                   Long        1  91
> < 20110626_213900.psd   Exif.Thumbnail.Compression                   Short       1  JPEG (old-style)
> < 20110626_213900.psd   Exif.Thumbnail.XResolution                   Rational    1  72
> < 20110626_213900.psd   Exif.Thumbnail.YResolution                   Rational    1  72
> < 20110626_213900.psd   Exif.Thumbnail.ResolutionUnit                Short       1  inch
> < 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormat         Long        1  366
> < 20110626_213900.psd   Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  0
8620a8735,8755
> > 20110626_213900.exv   Exif.Image.ImageWidth                        Short       1  150
> > 20110626_213900.exv   Exif.Image.ImageLength                       Short       1  91
> > 20110626_213900.exv   Exif.Image.BitsPerSample                     Short       3  8 8 8
> > 20110626_213900.exv   Exif.Image.Orientation                       Short       1  top, left
> > 20110626_213900.exv   Exif.Image.SamplesPerPixel                   Short       1  3
> > 20110626_213900.exv   Exif.Image.XResolution                       Rational    1  72
> > 20110626_213900.exv   Exif.Image.YResolution                       Rational    1  72
> > 20110626_213900.exv   Exif.Image.ResolutionUnit                    Short       1  inch
> > 20110626_213900.exv   Exif.Image.Software                          Ascii      30  Adobe Photoshop CS5 Macintosh
> > 20110626_213900.exv   Exif.Image.DateTime                          Ascii      20  2011:06:26 21:39:00
> > 20110626_213900.exv   Exif.Image.ExifTag                           Long        1  218
> > 20110626_213900.exv   Exif.Photo.ExifVersion                       Undefined   4  2.21
> > 20110626_213900.exv   Exif.Photo.ColorSpace                        Short       1  Uncalibrated
> > 20110626_213900.exv   Exif.Photo.PixelXDimension                   Long        1  150
> > 20110626_213900.exv   Exif.Photo.PixelYDimension                   Long        1  91
> > 20110626_213900.exv   Exif.Thumbnail.Compression                   Short       1  JPEG (old-style)
> > 20110626_213900.exv   Exif.Thumbnail.XResolution                   Rational    1  72
> > 20110626_213900.exv   Exif.Thumbnail.YResolution                   Rational    1  72
> > 20110626_213900.exv   Exif.Thumbnail.ResolutionUnit                Short       1  inch
> > 20110626_213900.exv   Exif.Thumbnail.JPEGInterchangeFormat         Long        1  366
> > 20110626_213900.exv   Exif.Thumbnail.JPEGInterchangeFormatLength   Long        1  0
Running geotag-test.sh ...
all testcases passed.
Running icc-test.sh ...
ICC jpg md5 webp md5 png md5 jpg md5 jp2 md5 all testcases passed.
Running imagetest.sh ...

Erase all tests..................
Copy all tests.........
Copy iptc tests.........
---------------------------------------------------------
All test cases passed
Running iotest.sh ...

file io tests
---------------------------------------------------------
All test cases passed
Serving HTTP on :: port 12761 (http://[::]:12761/) ...
::1 - - [13/Jan/2021 17:46:08] "GET / HTTP/1.1" 200 -
http://localhost:12761 status = 200
*** HTTP tests begin
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "HEAD /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "GET /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "HEAD /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "GET /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "HEAD /data/Reagan.tiff HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:09] "GET /data/Reagan.tiff HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:10] "HEAD /data/Reagan.tiff HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:10] "GET /data/Reagan.tiff HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:10] "HEAD /data/exiv2-bug922a.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:10] "GET /data/exiv2-bug922a.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "HEAD /data/exiv2-bug922a.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "GET /data/exiv2-bug922a.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "HEAD /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "GET /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "HEAD /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:11] "GET /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:12] "HEAD /data/table.jpg HTTP/1.0" 200 -
::ffff:127.0.0.1 - - [13/Jan/2021 17:46:12] "GET /data/table.jpg HTTP/1.0" 200 -
*** HTTP tests end
all testcases passed.
kill exiv2_httpServer 15578
Running iptctest.sh ...

Read tests..................
Remove tests..................
Add/Mod tests..................
Extended tests..................
---------------------------------------------------------
All test cases passed
Running iso65k-test.sh ...
all testcases passed.
Running nls-test.sh ...
1c1,2
< exiv2 not bulid with nls
---
> exiv2: Une action doit être spécifié
> exiv2: Se debe especificar una acción
Running modify-test.sh ...
all testcases passed.
Running path-test.sh ...
Running png-test.sh ...
all testcases passed.
Running preview-test.sh ...
.....................................
all testcases passed.
Running stdin-test.sh ...
stdin XMP a (exiv2, xmp, iptc) ICC
122,124d121
< ./functions.source: line 20: 17922 Segmentation fault      $VALGRIND $bin$prog$exe "$@"
< Exiv2 exception in insert action for file girl.jpg:
< Failed to read input data
129,138c126,137
<       20 | 0xffdb DQT   |      67
<       89 | 0xffdb DQT   |      67
<      158 | 0xffc0 SOF0  |      17
<      177 | 0xffc4 DHT   |      28
<      207 | 0xffc4 DHT   |      74
<      283 | 0xffc4 DHT   |      28
<      313 | 0xffc4 DHT   |      57
<      372 | 0xffe1 APP1  |    4613 | http://ns.adobe.com/xap/1.0/.<?x
<     4987 | 0xffe1 APP1  |    4602 | http://ns.adobe.com/xap/1.0/.<?x
<     9591 | 0xffda SOS
---
>       20 | 0xffe1 APP1  |    5704 | Exif..II*......................
>     5726 | 0xffe1 APP1  |    5287 | http://ns.adobe.com/xap/1.0/.<?x
>    11015 | 0xffed APP13 |     788 | Photoshop 3.0.8BIM..........Z...
>    11805 | 0xffdb DQT   |      67
>    11874 | 0xffdb DQT   |      67
>    11943 | 0xffc0 SOF0  |      17
>    11962 | 0xffc4 DHT   |      28
>    11992 | 0xffc4 DHT   |      74
>    12068 | 0xffc4 DHT   |      28
>    12098 | 0xffc4 DHT   |      57
>    12157 | 0xffe1 APP1  |    4602 | http://ns.adobe.com/xap/1.0/.<?x
>    16761 | 0xffda SOS
153,155d151
< ./functions.source: line 20: 17939 Segmentation fault      $VALGRIND $bin$prog$exe "$@"
< Exiv2 exception in insert action for file girl.jpg:
< Failed to read input data
160,169c156,167
<       20 | 0xffdb DQT   |      67
<       89 | 0xffdb DQT   |      67
<      158 | 0xffc0 SOF0  |      17
<      177 | 0xffc4 DHT   |      28
<      207 | 0xffc4 DHT   |      74
<      283 | 0xffc4 DHT   |      28
<      313 | 0xffc4 DHT   |      57
<      372 | 0xffe1 APP1  |    4613 | http://ns.adobe.com/xap/1.0/.<?x
<     4987 | 0xffe1 APP1  |    4602 | http://ns.adobe.com/xap/1.0/.<?x
<     9591 | 0xffda SOS
---
>       20 | 0xffe1 APP1  |    8528 | Exif..II*......................
>     8550 | 0xffe1 APP1  |    6730 | http://ns.adobe.com/xap/1.0/.<?x
>    15282 | 0xffed APP13 |     794 | Photoshop 3.0.8BIM..........Z...
>    16078 | 0xffdb DQT   |      67
>    16147 | 0xffdb DQT   |      67
>    16216 | 0xffc0 SOF0  |      17
>    16235 | 0xffc4 DHT   |      28
>    16265 | 0xffc4 DHT   |      74
>    16341 | 0xffc4 DHT   |      28
>    16371 | 0xffc4 DHT   |      57
>    16430 | 0xffe1 APP1  |    4602 | http://ns.adobe.com/xap/1.0/.<?x
>    21034 | 0xffda SOS
Running stringto-test.sh ...
all testcases passed.
Running tiff-test.sh ...
all testcases passed.
Running webp-test.sh ...
WebP delete insert ICC XMP EXIF all testcases passed.
Running write-test.sh ...
all testcases passed.
Running write2-test.sh ...
all testcases passed.
Running xmpparser-test.sh ...
all testcases passed.

---- Running python_tests ----

bash -c cd ../tests; python3 runner.py
........Serving HTTP on 127.0.0.1 port 12760 (http://127.0.0.1:12760/) ...
127.0.0.1 - - [13/Jan/2021 17:50:28] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:28] "HEAD /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:28] "GET /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /Reagan.tiff HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /Reagan.tiff HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /Reagan.tiff HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /Reagan.tiff HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /exiv2-bug922a.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /exiv2-bug922a.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /exiv2-bug922a.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /exiv2-bug922a.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "HEAD /table.jpg HTTP/1.0" 200 -
127.0.0.1 - - [13/Jan/2021 17:50:29] "GET /table.jpg HTTP/1.0" 200 -
....Skipped. Because exiv2 is not built with nls.
............make[4]: *** [Makefile:157: python_tests] Interrupt
make[3]: *** [CMakeFiles/tests.dir/build.make:77: CMakeFiles/tests] Interrupt
make[2]: *** [CMakeFiles/Makefile2:364: CMakeFiles/tests.dir/all] Interrupt
make[1]: *** [CMakeFiles/Makefile2:371: CMakeFiles/tests.dir/rule] Interrupt
make: *** [Makefile:209: tests] Interrupt
heckflosse commented 3 years ago

@kmilos

What's wrong with -G"MSYS Makefiles"?

Nothing, we also use this for RawTherapee. The one thing which did not work, was make install, but after adding -DCMAKE_INSTALL_PREFIX="$MSYSTEM_PREFIX" to the cmake command it worked fine.

clanmills commented 3 years ago

I've reproduced the "girl.jpg" issue. I recently changed the girl.jpg file. I will investigate. I'll have a look at 20110626_213900.psd. That feels familiar, but I don't remember the story at the moment.

You'll see that you experiences a crash on the Valgrind thing. That's a "happens now and then" issue. Can you try rerunning the tests. Both Cygwin64 and MinGW are quite prone to the VALGRIND message. I think it has to do with position dependent code or something.

heckflosse commented 3 years ago

Can you try rerunning the tests. Both Cygwin64 and MinGW are quite prone to the VALGRIND message

I can not remember that I ever have installed Valgrind in msys2. Maybe the test just crashed because it's not installed. It didn't even know, that I can install it on msys2. I always use Linux in a VM to valgrind. Maybe valgrind should be disabled for windows/msys2 make tests?

clanmills commented 3 years ago

Oh, you don't need valgrind. It's just the message that mentions it:

< ./functions.source: line 20: 17922 Segmentation fault      $VALGRIND $bin$prog$exe "$@"

I've remembered why I haven't investigated. It's coming from the bash_tests which have been rewritten in python. I haven't removed bash_tests, however it's on the list to be done: #1274. You'll see in appveyor_ming_cygwin.yml, I only run the python_tests to keep away from those false negatives:

515 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $ grep tests appveyor_mingw_cygwin.yml 
    - cmd: set   CMD=%CMD%; cd build ; cmake --build . --config Release --target python_tests
    - cmd: set  CMD=%CMD%; make python_tests
516 rmills@rmillsmm-local:~/gnu/github/exiv2/0.27-maintenance $

So, I think we're good here and don't need to do anything special. I should deal with #1274. I was leaving that for the end just in case some serious difference between the old and the new tests appears.

kmilos commented 3 years ago

Btw, using -GNinja fails on mingw on the final, linking step.

Adding AND MINGW here seems to solve it (or potentially changing MSYS to MINGW), and I could build any combo of gcc/clang compiler and Ninja/MSYS Makefile generator.

FYI: -G"MSYS Makefiles" seems to define MSYS and MINGW (elsewhere), while only MINGW is defined when generating w/ -GNinja... So you end up having -D_FORTIFY_SOURCE=2 in the compile options but without -fstack-protector-strong (excluded in the case above using only MINGW), while it seems it is a prerequisite. See also https://github.com/msys2/MINGW-packages/issues/5868#issuecomment-544107564

If you're ok with this, I can make a PR.

clanmills commented 3 years ago

I'm not sure what you're saying, so please submit a PR.

As I said yesterday, we cannot support every possible way to build, so provided this change doesn't break the documented build (which is used by the CI), then I'm OK about this. I don't see any reason to modify README.md about this.

kmilos commented 3 years ago

Yep, no need to change anything, just improves overall platform coverage.

clanmills commented 3 years ago

By the way, the crash in Cygwin/64 and MinGW/msys2 occurs in the script and not in the exiv2 application. The "new" bash tests are written in python, however the functionality is the same as before. They execute the same commands and use the same reference files. Changing to python has been done to make it easy to test of Visual Studio builds as there is no dependency to have bash, make and other linux utilities on the platform.

I haven't totally understood why the old bash tests perform this "Seg Fault". My suspicion is the pressure on the stack as the bash is running make which runs another bash ...bla bla... Life from the python wrapper is simpler. Python is running and it runs exiv2 and captures the output. For sure it doesn't SegFault when creating a process to run exiv2.

heckflosse commented 3 years ago

Currently, if I want to contribute, I would have to use the 0.27-maintenance branch because I'm on msys2 and master does not build on msys2. That means, I would need to make pull requests against 0.27-maintenance instead of master branch which does not sound good to me as 0.27-maintenance is way behind master branch but also way ahead of master branch.

@clanmills which branch shall I use for contribution?

clanmills commented 3 years ago

Good to hear your voice. Welcome back. I've granted you write access to exiv2/exiv2. By submitting PR's directly to that repos, I can edit/modify your PR if necessary.

Please use 0.27-maintenance. Many changes from master (for sure all security fixes) are in 0.27-maintenance. 0.27-maintenance will ship on 2021-05-22 as 0.27.4.

We split off 0.27-maintenance when we shipped 0.27 in December 2018. Important work has been done on master to replace AutoPtr with UniquePtr and lots of other C++11 (and later) changes. The plan was for me to release "dots" to keep Exiv2 up-to-date while Dan and Luis "modernised" the code-base. Regrettably, due to life (work, partners, covid) Dan and Luis haven't completed their work. I haven't contributed to 'master', so I don't know the status of the branch. This situation is disappointing, however it's not bad. The good news is that Exiv2 0.27 is maintained and supported.

Since 0.27.3 shipped on 2020-06-30, there has been a steady increase in contributors. So, I'm optimistic that 'master' will be completed in 2021 and ship as 0.28. Exactly how this will happen is unclear. I've been talking to @alexvanderberkel about having a "Team Exiv2" meeting in England when travel restrictions ease in late summer. A physical meeting could really get the contributors to pull together and see 0.28 finally appear.

I did investigate the message:

< ./functions.source: line 20: 17922 Segmentation fault      $VALGRIND $bin$prog$exe "$@"

on static builds and shared+PIC enabled. The crashes in the bash tests are puzzling. If you have ideas, I would appreciate your input and thoughts. Or we ignore this as the new python tests are working well.

clanmills commented 3 years ago

@heckflosse I'm going to close this as I don't intend to do further work concerning this. If you have new thoughts about this, I'll be happy to re-open the issue.