Exiv2 / exiv2

Image metadata library and tools
http://www.exiv2.org/
Other
911 stars 280 forks source link

Is a new release happening soon? #2406

Closed SuperSandro2000 closed 1 year ago

SuperSandro2000 commented 1 year ago

exiv2 has currently some medium to high opne CVEs in the latest release (like this 9.8 https://nvd.nist.gov/vuln/detail/CVE-2022-3717 ) which should be patched rather quickly in distros and software distributions. Normally applying a patch for the fixes is rather easy but because of the big formatting patches (see https://github.com/Exiv2/exiv2/blob/main/.git-blame-ignore-revs) the patches/commits no longer apply on the latest release.

Are there any plans for a new release in the new future? If a release is far in the future maybe the patches could be rebased on the latest release and a new security fixes only release could be done?

risicle commented 1 year ago

I'd like to add to this a bit of feedback on having actionable vulnerability reports. Regarding the 6 recently released CVEs:

https://nvd.nist.gov/vuln/detail/CVE-2022-3717 https://nvd.nist.gov/vuln/detail/CVE-2022-3718 https://nvd.nist.gov/vuln/detail/CVE-2022-3719 https://nvd.nist.gov/vuln/detail/CVE-2022-3755 https://nvd.nist.gov/vuln/detail/CVE-2022-3756 https://nvd.nist.gov/vuln/detail/CVE-2022-3757

For each of these the suggested course of action is patching, but not a single one of the indicated patches apply to the most recent stable release without significant alteration. Any such alteration would also be a rather blind process because the proof of concepts I could find for these issues don't seem to trigger v0.27.5, so it would be hard to verify that it's been fixed.

It's possible these aren't triggering v0.27.5 because of a recent slightly mysterious refactor of the BMFF support that was done in August: https://github.com/Exiv2/exiv2/commit/3456f3098897330ea3a3efff9b19f624e407c22e which wasn't accompanied by any announcement.

CVE-2022-3717 is of special note because I don't think it affected any versions outside that PR, so I'm not quite sure what the purpose of that was.

So it's difficult for a distributor/user to know what to do now. If the answer is "run an unstable version" (which right now looks like it might be the only sensible option for shipping a safe exiv2), then you may as well make an immediate release because I guess at this point any QA criteria go out the window.

postscript-dev commented 1 year ago

Are there any plans for a new release in the new future?

After Robin retired from Exiv2, @nehaljwani kindly offered to take over release engineering. The transition is still relatively new and will take time to work smoothly.

As far as I know, @nehaljwani doesn't follow the day-to-day posts but can be contacted by mentioning him.

clanmills commented 1 year ago

I will help @nehaljwani and anybody to make a release. I'm pleased to be retired and will not return to the project. I am happy to see Exiv2 move forward and will help when asked.

Please email me if you need my help, as I have unsubscribed from Github notifications.

clanmills commented 1 year ago

@nehaljwani and I are meeting today (2022-11-12) on Google Meet to discuss this. All welcome.

Robin/Nehal Saturday, 12 November · 14:00 – 15:00 UTC Google Meet joining info Video call link: https://meet.google.com/mfk-zpii-gic

If you want an invitation, email me: at robinwmills@gmail.com

clanmills commented 1 year ago

The most important matter in this release is the scope and version. Choices:

  1. Release 'main' as version 0.28.0 (or possibly 0.90.0 to make it very clear that this is not a simple successor to 0.27)
  2. Release 'main' as version 1.00.0 (I don't think 'main' is v1.00.0)
  3. Patch v0.27.5 with the security fixes and release v0.27.6

I don't know the status of the 0.27-maintenance branch. If security fixes have to be back-ported to branch 0.27-maintenace, the work involved could be considerable.

nehaljwani commented 1 year ago

Hello everyone! Robin has shared with me the steps involved in cutting a release.

IIUC, the main branch diverged from 0.27-maintenance right after 0.27.4 with this commit.

If @kevinbackhouse can confirm that the required security fixes are present in the 0.27-maintenance branch (or not needed because of irrelevance), we can proceed with a v0.27.6. As for a release from the main branch, I vote for v0.75.0.

I invite @postscript-dev, @kmilos, @piponazo, @neheb to voice their opinion.

I hope to cut the releases by the end of this month with the versions listed above if no concerns are raised.

postscript-dev commented 1 year ago

@nehaljwani Thanks for taking the time to work on this. As I build from source, the new release doesn't affect me much.

I had a couple of thoughts on the release process.

I recently spotted that files used when building the website are duplicated (e.g., https://github.com/Exiv2/exiv2/blob/main/doc/templates/Makefile and https://github.com/Exiv2/team/blob/main/website/Makefile). As far as I know, the Exiv2/team ones are not being kept up to date. There are some differences in Exiv2/Exiv2/docs/templates - particularly on the main branch.

Also on the main branch, the manpage has changed to markdown format (i.e.Exiv2/Exiv2/man/man1/exiv2.1 to Exiv2/Exiv2/exiv2.md). As a result, it wont be automatically added to https://exiv2.org/manpage.html. The manpage text may also need minor reformatting to fit any line limits used by the postscript file. If you need help with the manpage text, then let me know.

It is worth noting that the main branch has diverged a LOT from the commit that you named. For 0.75.0 , apart from the size of the release notes, there could be other minor changes needed to the release process. As a release is needed, perhaps it would be better to complete 0.27.6 first and then work on 0.75.0 after.

piponazo commented 1 year ago

Hi everybody! I have not been able to contribute much to the project lately, but I still keep to keep an eye on what's going on and deal with the CI problems when it is needed. Since winter is coming and is full of terrors, I will probably show up more here 😄 .

For me it also sounds like a good plan to make 2 releases:

Whenever we think that main is in a state mature enough we can think about 1.0.

kevinbackhouse commented 1 year ago

Sorry, I haven't been paying to this issue. I'm quite angry to discover that a bunch of CVEs have been filed without consulting any of us. (I very much doubt it was done by anybody on the Exiv2 team, because we would have used GitHub Security advisories instead.) I haven't checked them all yet, but I'm pretty confident that all of those CVEs are bogus because they were introduced on the development branch (main) and fixed within a few weeks. As an example, CVE-2022-3717, which is cited as the main example in this issue, was fixed as the 8th commit in #2381, so it never even got merged into the main branch.

We have a security policy which spells our very clearly that bugs on the main branch are not security bugs. Only bugs in official releases, such as v0.27.5 are potential security vulnerabilities. I have been paying attention to all the potential security issues and I'm pretty confident that none of the bugs found recently have been reproducible on the 0.27-maintenance branch.

I will contact the CNA to dispute those CVEs and get them removed.

kevinbackhouse commented 1 year ago

Regarding a new release, I am in favor of doing it soon. I think we should do a v0.27.6 and I think it's time to do a v1.0.0 too. My own main goal for 1.0.0 was to replace all the uses of the long type with something more appropriate, such as size_t. I think that's mostly done - I think there are still some uses in some of the .cpp files, but the header files are mostly clean now so the remaining problems can be fixed without affecting the public API.

1div0 commented 1 year ago

WRT v1.0.0 I do hope Big Blue Red Hat lawyers will at least state in https://bugzilla.redhat.com/show_bug.cgi?id=1979565 that your camera means your data and no fooced shitware patents apply.

kmilos commented 1 year ago

Thanks for stepping up @nehaljwani 👍

If it plugs all the known security holes, I'm in favour of 0.27.6 asap, as it has some useful added functionality and bug fixes backported.

Re the next version, I don't really care what it's called, as long it comes soonish as well (there are projects depending on exiv2 that are slowly starting to require at least C++17). One thing that needs to be sorted out though is the SO version mess (from that vantage point, I'm even ok w/ 0.28.0...)

kevinbackhouse commented 1 year ago

Those bogus CVEs have been rejected now. For example: https://nvd.nist.gov/vuln/detail/CVE-2022-3717

postscript-dev commented 1 year ago

@nehaljwani Support for video was removed from the main branch however, some users requested this to be restored. @mohamedchebbii has worked hard to recover and modernise the code to be compatible with C++17. It would be good to include this in the new main release, if possible. @mohamedchebbii, how close is your work to being merged?

mohamedchebbii commented 1 year ago

Hello @postscript-dev , I pushed my three pull requests and they seems good for me: https://github.com/Exiv2/exiv2/pull/2413 https://github.com/Exiv2/exiv2/pull/2415 https://github.com/Exiv2/exiv2/pull/2416

Please let me know, If I need to fix or add somthing else ? Regards,

postscript-dev commented 1 year ago

@mohamedchebbii Thanks for the update. As I don't have much free time, I was planning to leave your review to someone else.

benmccann commented 1 year ago

The video support has been merged. Since it's a major feature, maybe the next release should be 0.28.0? It would be great to be able to get it out there. We're stuck on an outdated version of exiv2 (the last one with video support before it was removed), and are really looking forward to upgrading soon :smile: Thanks to @mohamedchebbii for all the great work on that feature and thanks to @piponazo for the reviews!

kevinbackhouse commented 1 year ago

As I just commented in #2450, I would like to roll back the recently merged video support until it has been better tested.

I think Exiv2 was in very good shape before those PRs were merged, so my suggestion is that we start preparing 0.27.6 and 1.0.0 releases based on the state of the code as it was on 2022-12-30.

postscript-dev commented 1 year ago

@nehaljwani I was reading some closed issues and was reminded of something. In regards to a new main release, we now require a minimum of Windows 10 for the command line programs (#2090). This applies to the exiv2 app and sample programs that we include in the download. This caveat needs to be added to the Exiv2 website as part of the release.

neheb commented 1 year ago

According to https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-170 , prior Windows versions can be supported if libc is statically linked.

nehaljwani commented 1 year ago

Exiv2 v0.27.6 2023-01-18

Topic More Info
Remaining ( 0) https://github.com/Exiv2/exiv2/milestone/10
Completed (41) https://github.com/Exiv2/exiv2/milestone/10?closed=1

Exiv2 v0.27.6 Acknowledgements

According to git shortlog -s -n v0.27.5..v0.27.6 Commits Author
30 Miloš Komarčević
20 Robin Mills
13 Luis Díaz Más
11 Kevin Backhouse
4 Luis Diaz
4 Nehal J Wani
3 Peter S
1 Christoph Hass

Exiv2 v0.27.6 Release Notes (updated 2023-01-16)

Group PR Topic Issue
Documentation #2404 Update docs concerning BMFF-based files #2398
#2328 Fix rename text in manpage #2290
Lens #2315 Add Nikon3.WhiteBalanceBias2
#2291 Add Nikon LensData v0802
#2243 Add some F mount lenses #2193, #2196, #2214
#2167 Initial support for OM System MakerNote #2126
#2046 Add Sony ARW compression to dict
Build #2315 Upgrade conan to version 1.51.0
#2241 Fix CI jobs - Update conan packages
#2142 Update CI actions
#2140 Update Windows CI workflows
#2033 Updating documentation to respect ctest
#2032 Switch Cygwin CI to GHA
#2025 Nightly pre-builds for 0.27 #1984
#2024 Running tests with ctest #2022
#2019 Switch MinGW CI to GHA
#2018 Fix conan ci
#2004 Streamline MinGW package installation for CI
#2003 Fix broken Cygwin/CI
#2002 Fix CMake build type #1558
Bugs & Fixes #2382 Exif start can be at any byte in payload, not word aligned #2381
#2375 Fix exception type when writing BMFF files #2350
#2333 Add more MIME type mappings for TIFF-based raws #2260
#2261 Fix naming of canon EF 35-80mm #2247
#2269 Replace assert with enforce #2268
#2256 PNG: always strip the existing iCCP chunk
#2239 Account for header bytes for Exif and XMP boxes #2233
#2194 Fix Integer overflow in Photoshop::setIptcIrb #2179
#2192 Fix Integer-overflow in sumToLong #2190
#2186 Fix out of bounds read in isValidBoxFileType() #2178
#2153 Fix in Jp2 metadata writing & improvements in reading #2147
#2139 Strip XMP raw packet before decoding #2126
#2045 Add tiff tags #2044
#2026 Add more DNG 1.6 tags
#2037 Fix bug in iterating over the elements of dateStrings #2036
#2030 Use memmove in TiffEncoder::updateDirEntry #2027
#2016 Treat Exif.Sony1.PreviewImage as undefined tag #2001
Tests #2269 Regression test for Exiv2#2268 #2268

If I've failed to acknowledge anyone's contribution, I apologize. Please let me know and I'll update this comment.

kmilos commented 1 year ago

Thanks! @nehaljwani

Since this is also driven by security needs, maybe it's worth listing what CVEs have been plugged in the release notes as well? @kevinbackhouse

kevinbackhouse commented 1 year ago

Thanks! @nehaljwani

Since this is also driven by security needs, maybe it's worth listing what CVEs have been plugged in the release notes as well? @kevinbackhouse

No new CVEs in this release. All known security bugs were caught during development and never made it into an official release.

kmilos commented 1 year ago

All known security bugs were caught during development and never made it into an official release.

I just re-read your response a while back - so no valid CVEs against 0.27.5, and the release notes look good then. 👍

nehaljwani commented 1 year ago

Dear folks, Exiv2 v0.27.6 has been released!

... 1.0.0 releases based on the state of the code as it was on 2022-12-30.

I'll start working on the v1.0.0 major release based on https://github.com/Exiv2/exiv2/commit/9ca161d1e5696623a2a3b97860458f90ed01c511

Please raise concerns, if any.

kmilos commented 1 year ago

I'll start working on the v1.0.0 major release based on 9ca161d

Please raise concerns, if any.

See again https://github.com/Exiv2/exiv2/issues/2406#issuecomment-1322291596

jim-easterbrook commented 1 year ago

Do you plan to add a Source download to the Linux, Darwin and msvc? The Download Source button on the exiv2.org download page is currently broken.

postscript-dev commented 1 year ago

@nehaljwani I noticed that the release notes for 0.27.6 are missing on https://exiv2.org/whatsnew.html . Hopefully a simple fix.

1div0 commented 1 year ago

@nehaljwani Thank you so much.

So far so good.

While experimenting with the various build configurations by enabling the NLS support, I have noticed the following:

` CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Failed to find Gettext libintl (missing: Intl_LIBRARY) Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE) /usr/share/cmake/Modules/FindIntl.cmake:161 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) cmake/findDependencies.cmake:70 (find_package) CMakeLists.txt:77 (include)

Configuring incomplete, errors occurred! See also "/2TB/usr/src/github.com/1div0/exiv2/Linux/x86-64/CMakeFiles/CMakeOutput.log". See also "/2TB/usr/src/github.com/1div0/exiv2/Linux/x86-64/CMakeFiles/CMakeError.log". ` This is on F37, everything up to date. Is it already known issue?

My free(time_t) is very limited, will get back to topic by the WE.

jim-easterbrook commented 1 year ago

The Download Source button on the exiv2.org download page is currently broken.

It's OK now! Thanks for the quick fix.

benmccann commented 1 year ago

Will 1.0 and 0.27.6 be essentially the same release? If so, what's the point of having the same release with two different version numbers? Maybe 1.0 should include some big exciting new features like video support?

SuperSandro2000 commented 1 year ago

Failed to find Gettext libintl (missing: Intl_LIBRARY)

Missing gettext I would guess.

If so, what's the point of having the same release with two different version numbers?

To promote a stable release and know it is stable. An alternative would be to do a beta or rc candidate first.

benmccann commented 1 year ago

If we want to treat 0.27.6 as a beta or RC, maybe we'd want to let more time pass so that we're able to collect some bug reports?

kmilos commented 1 year ago

If we want to treat 0.27.6 as a beta or RC, maybe we'd want to let more time pass so that we're able to collect some bug reports?

We don't want to do that. The main branch (from which "1.0.0" or whatever-the-next-version-number will be derived) has significant refactoring and changes to the codebase and is to be considered very different from the 0.27.x series (although feature-wise, indeed similar at this point in time). And we don't want to actively develop/support 0.27.x further either (it's been around for 4 years now).

benmccann commented 1 year ago

Ah, I didn't realize there was a different branch for 0.27. Thanks for clarifying

1div0 commented 1 year ago

@kmilos Can't agree more. For one divided by zero release there needs to be clear way forward regarding the BMFF. For all fancy new formats as well as renewed video support.

clanmills commented 1 year ago

Well done, @nehaljwani, for the 0.27.6 release on 2023-01-18, which was my 72nd birthday. Thank You, Nehal and everybody, for getting this done.

1div0 commented 1 year ago

Failed to find Gettext libintl (missing: Intl_LIBRARY)

Missing gettext I would guess.

/usr/share/cmake/Modules/FindIntl.cmake

.. note:: On some platforms, such as Linux with GNU libc, the gettext functions are present in the C standard library and libintl is not required. Intl_LIBRARIES will be empty in this case.

This seems to be the case.

F37 now, I will check RHEL and Rocky ASAP.

1div0 commented 1 year ago

Rocky Linux 8.7 (Green Obsidian)

[peter.kovar@xn--dsseldorf-q9a Test]$ LANG=sk_SK.UTF-8 exiv2 * IMGP0191.jxl Názov súboru : IMGP0191.jxl IMGP0191.jxl Veľkosť súboru : 684603 bajtov IMGP0191.jxl MIME typ : image/generic IMGP0191.jxl Veľkosť obrázka : 3040 x 2024 IMGP0191.jxl Náhľad : Nič IMGP0191.jxl Zmačka fotoaparátu: PENTAX Corporation IMGP0191.jxl Model fotoaparátu: PENTAX K100D
IMGP0191.jxl Časová známka obrázka: 2019:09:21 21:41:07 IMGP0191.jxl File number : IMGP0191.jxl Čas expozície : 0.4 s IMGP0191.jxl Clona : F3.5 IMGP0191.jxl Skreslenie expozície: 0 EV IMGP0191.jxl Blesk : No, compulsory IMGP0191.jxl Skreslenie blesku: IMGP0191.jxl Ohnisková vzdialenosť: 18.0 mm IMGP0191.jxl vzdialenosť subjektu: IMGP0191.jxl ISO rýchlosť : 800 IMGP0191.jxl Režim expozície : nedefinované IMGP0191.jxl merací režim : Multi-segment IMGP0191.jxl režim makro : IMGP0191.jxl Kvalita obrázka : IMGP0191.jxl Vyváženie bielej: Auto IMGP0191.jxl Copyright : IMGP0191.jxl Komentár EXIF :

Almost perfect, with the one small exception: MIME type is reported as image/generic instead of the image/jxl.

@kmilos ?

1div0 commented 1 year ago

@kmilos it is correct in the main, so let's focus on 1.0 release.

postscript-dev commented 1 year ago

1div0 wrote: For one divided by zero release there needs to be clear way forward regarding the BMFF. For all fancy new formats as well as renewed video support.

I don't use BMFF files and haven't studied the legal arguments in detail.

However, I agree that resolving the BMFF issue would help Exiv2 move forward. One way to achieve this would be to join the Open Invention Network (OIN) - as discussed in #1447. In the first post, there is a list of benefits to joining but I think the following reasons also apply.

  1. We can bring an end to the BMFF issue and provide a common way forward.
  2. Once the BMFF issue is resolved, we are free to concentrate on other issues.
  3. After joining, we can fully enable BMFF helping many distributions, projects and users.

If there is agreement amongst the Exiv2 team, then I think joining only takes one of the maintainers a few minutes to sign.

After joining, the Exiv2 code could then be simplified on the main branch to enable BMFF for everyone. If joining is agreed upon quickly, we might even be able to include this in the next release.

It would be helpful to know what everone else thinks about this. Further discussions on joining the OIN can take place in #1447.

kmilos commented 1 year ago

It would be helpful to know what everone else thinks about this.

I'd just simply drop this - Exiv2 already had BMFF parsing code in jp2image and quicktimevideo for ages.

kmilos commented 1 year ago

@1div0 It was actually your patch that didn't get back-ported 😞

1div0 commented 1 year ago

I am so sorry about that. Really, multitasking is affecting me badly.

kevinbackhouse commented 1 year ago

I'll start working on the v1.0.0 major release based on 9ca161d Please raise concerns, if any.

See again #2406 (comment)

This thread is quite long, so just chiming in to say that I agree with @kmilos that 9ca161d would be a great candidate to create the v1.0.0 release from.

kmilos commented 1 year ago

9ca161d would be a great candidate to create the v1.0.0 release from.

Apart from the SOVERSION problem, I'm having some doubts re the version: if this is the branch of point, that means no video support, and no external inih dependency - presumably those could come back in the near future, and that is not really indicative of a stable "1.0" product IMHO (sure, one could bump to 2.0)... Let alone the fact that there are still some pretty significant TODOs on the 1.0 list we never managed to implement... So, I vote for the 0.28 branch and 0.28.0 stabilization.

benmccann commented 1 year ago

Video support has been merged. Would it be possible to get a new release which includes it?

1div0 commented 1 year ago

Thank you so much.

Everything is possible.

We need to make 120% sure it would not introduce any vulnerability by extensive testing on various large datasets.

Definitely doable.

benmccann commented 1 year ago

Yes, I definitely agree we should make sure video support and the library in general are well tested.

I do want to highlight that there's also risk in holding back a release for an extended period of testing. A lot of users are stuck on the last version of exiv2 with video support and can't upgrade until we do a release. E.g. all Synology NAS devices come with exiv2 0.27.3. Synology Photos uses it for photos and videos. There's a huge number of people stuck on older versions with vulnerabilities that have already been fixed. Until we release a new version they're not going to be able to upgrade to get any of the fixes. Even if we were to release with some vulnerabilities it would still be a huge improvement for a vast set of users and would greatly cut down on the security risks they're being exposed to. Hopefully we can make the testing period quick and release a new version without too much delay.

1div0 commented 1 year ago

Agree, but TANSTAAFL.