New major release - RT5

[Beep6581]

This is an umbrella issue for tracking issues which need to be solved so that we can release a new major version. A new release is so overdue that we should strive to get RawTherapee in shape as soon as possible, therefore these issues should not be things we would like solved but things we need solved.

• [x] #1791 Automatically disable Auto-fill when enabling LCP
• [x] #1812 Persistent color picker tool / Lockable Color Picker
• [x] #2568 tunnelMetaData IPTC, XMP and/or Exif?
• [x] #3048 Segmentation fault when switching to next/previous image while curve pipette is enabled
• [x] #3186 Combobox text overflow gtk3
• [x] #3253 Speedup for Flat-Field correction
• [x] #3275 Merge branch rgbcurvesspeedup - Speedup for thumbnail processing
• [x] #3302 dcraw 9.27
• [x] #3304 Race when using same dcp profile with different settings
• [x] #3331 Speedup for dcp processing
• [x] #3334 Update rtexif with latest exiftool data
• [x] #3346 Rework median defines - median speedup
• [x] #3360 Fix Denoise - Median tooltip
• [x] #3367 Segfault when keeping MakerNote and save to tiff16 uncompressed
• [x] #3370 Exif lost when saving compressed tiff from raw on windows builds
• [x] #3406 Soft-proofing branch
• [x] #3443 Fix buffer overflow in dcraw
• [x] #3446 Add support for Gtk+ 3.20 and/or 3.22
• [x] #3465 Clang-tidy
• [x] #3473 CbDL badly interacts with Graduated and Vignetting Filter if CbDL is set to 'before Black-and-White'
• [x] #3474 Segfault on EAHD
• [x] #3480 Crop-guide is wrong when panning image out of preview in Fit-To-Screen mode
• [x] #3483 Gtk+ >=3.20 Comboboxes must not let content overflow
• [x] #3500 Gtk+ >=3.20 Meta tab Exif unusable
• [x] #3506 Navigator/LockableColorPicker does not show correct RGB/HSV for output profile
• [x] #3590 DCP batch settings broken

Code will be astyled before tagging.

[Beep6581]

I removed "#3264 Merge branch newwavelet" so that RT5 gets out sooner and so that we have more time and less stress to complete and test the newwavelet branch.

[Beep6581]

The gtk3 branch has been well tested using Gtk+ 3.18, and it works generally great except for #3223 - Gtk::FileChooserButton are slow to load. I suggest releasing RT5 with a gtk3-branch dependency requirement on Gtk+ 3.16 - 3.18. We leave 3.20 support for RT-5.1, not only because it would require weeks or months for patching and testing but also because most distros don't even ship it yet. Any objections?

[heckflosse]

One objection: As I'm still building master (gtk2) on windows, I need some help to build gtk3 an windows to avoid outage

[Beep6581]

Making gtk3 the new master does not have to happen when 5.0 is tagged. My suggestion was just about making the RT-5.0 gtk3 branch have Gtk+ 3.16 or 3.18 as a requirement - that is we agree not to support 3.20 until after 5.0 is tagged.

[heckflosse]

Ok, then I agree :+1:

[Hombre57]

@heckflosse If you need help with the gtk3 branch on Windows, just drop me a word by email when you're ready to experiment.

[Hombre57]

So if I understand correctly, the soft-proofing branch is one of the last issue to fix... :-/ I'll have to put the Locked Color Picker patch on hold and finish the Soft-proofing feature first. Do you agree to have a suboptimal soft-proof feature ? i.e. something stable and better than today's situation but still unfinished ?

[Beep6581]

I think that would be fine. Reasons:

1. It would remove stress from you,
2. It would enable wider testing,
3. Tuning the code to work perfectly for both print proofing and output proofing could take months, this would encourage a small-steps/RERO approach.

[heckflosse]

I added #3048

[innir]

Please apply PR #3443 before releasing RT 5. it fixes a buffer overflow in dcraw.

[heckflosse]

@innir I added it to the checklist above

[heckflosse]

@Hombre57

@heckflosse If you need help with the gtk3 branch on Windows, just drop me a word by email when you're ready to experiment

Problem solved by using different folders for building master and gtk3 :)

[Hombre57]

Great ! Now we have to make the new theme work on Windows again, and I have to finish the soft-proofing branch.

[heckflosse]

@Hombre57 I think we should give priority to rt5 over an optimized softproofing branch. That means, if you agree, I'll look for optimizing softproofing code after rt5 is released. Is that ok?

Ingo

[Hombre57]

@heckflosse Sure, and I'll only do a first level soft-proofing feature, regarding the comments above.

[heckflosse]

@Hombre57 Great :+1:

[heckflosse]

@Hombre57

Now we have to make the new theme work on Windows again

Which is the 'new theme'? I like the 'TooWaBlue' theme using gtk3 branch on Win7. But I don't know whether there are issues using it.

Ingo

[Beep6581]

Morning

Now we have to make the new theme work on Windows again

What do you mean? Gtk+ 3.16 and 3.18 are supported, 3.20 is not. Which version of Gtk+ did you use and which theme are you referring to?

[Floessie]

@Beep6581 @innir Debian Stretch now has GTK 3.22. Will that be supported?

[Beep6581]

@Floessie Yes, that's the plan, but I can't compile it in Gentoo yet because it would break many other programs.

[Floessie]

@Beep6581 Good then! Qemu/KVM to the rescue?

[Beep6581]

That would be one option, but relatively few people have access to 3.22 making it medium priority. My high priority is the RT website, in dire need of dynamite and nails.

[Hombre57]

What do you mean? Gtk+ 3.16 and 3.18 are supported, 3.20 is not. Which version of Gtk+ did you use and which theme are you referring to?

I'm using Gtkmm3.22 as that's the one provided by MSYS2 now, and I'm talking about all the bundled theme.

[innir]

@Floessie Yes, Debian Stretch will ship with GTK+ 3.22. RT in Debian unstable is already built with GTK+ 3.22 and it works fine (from what I see). To be sure to get RT 5 in Debian stretch it should be released latest end of the year.

[Beep6581]

It would be nice if RT5 could support Gtk+ 3.22... Yesterday I compiled the latest Gtk+ available on Gentoo, that's 3.20.9. If 3.22 is compatible with 3.20, then we can kill two birds with one stone. I want RT5 out very soon, this month. If 3.20/3.22 support doesn't make it into RT5(.0), then by the end of the year we could release a 5.1.

[innir]

@Beep6581 I already built RT with Gtk+ 3.22, it's in Debian unstable so you could give it a try. I didn't see any obvious problems.

[Beep6581]

@innir thank you but I don't use Debian and I make my own builds.

[Beep6581]

TODO when releasing:

• http://rawpedia.rawtherapee.com/How_to_release_RawTherapee
• astyle code
• Replace (c) and (C) with © (or whatever, to keep things consistent. #3445)
• Update versions (4.2?) and dates
• Add links to new forum to About window
• Summarize new features significant to a user instead of a detailed changelog
• Release Notes, add comment about building for Windows from https://github.com/Beep6581/RawTherapee/issues/3544#issuecomment-267606005
• Coverity
• Bulk cleanups with clang-tidy (@Floessie)
  • modernize-use-nullptr
  • readability-braces-around-statements
  • modernize-redundant-void-arg
  • readability-simplify-boolean-expr
  • misc-redundant-expression
  • readability-redundant-control-flow
  • readability-avoid-const-params-in-decls

[Beep6581]

If anyone has things they'd like to do, or things they'd like me to do, just before releasing RT5, then feel free to make a list or to edit my list above.

[Hombre57]

I'll look at the Gtk3.22 theme ASAP, hopefully before the end of next Sunday. If I can't fix it, then you'll have to release a Gtk3.18 Windows version.

[Beep6581]

@Hombre57 ok, I added #3446 to the list so we don't forget. I was unexpectedly upgraded to Gtk+ 3.20.9 a few days ago.

[sguyader]

@Beep6581 I would like to see #3229 fixed with the next RT release. I think a number of advanced or pro photographers turn to the latest Fujifilm cameras, I think complete support of their raw files would make RT more attractive to them, as RT is now known as one of the best raw developers for X-trans files.

[heckflosse]

@sguyader Hi Sébastien. I already started work at porting the libraw code for compressed xtrans files to rt. But it's a lot of work because it has many dependencies to libraw internals. For that reason I would like to complete it after rt5 is released. @Beep6581 already planned to release 5.1 end of the year. That would fit much better (for me at least) ;-)

[sguyader]

@heckflosse ok Ingo, I had no idea it requires so much work to get the libraw code into RT.

[Beep6581]

Hello

re: https://github.com/Beep6581/RawTherapee/commit/ccd9002c3abc504e3a25b7bbf0489b90b4cf808e#commitcomment-19352462 I like fixes, fixes are good. I'm very glad you guys have been focusing on fixes for so long - RawTherapee is much better off thanks to them.

Generally what one would do at this point is to branch off and make release candidates and then a final release when ready. I could do that, but it would require two branches: a master-rt5 and gtk3-rt5 branch, and then tags in each. A bit messy. But there are only a few of us, and as RT doesn't get new features often anyway, it would be simpler to just agree not to commit anything but fixes for a week or two. Would that work for you? Or do you have strong arguments for branching off?

Committing those cppcheck etc fixes would make it easier to maintain other open branches and patches. I understand there are risks - merging them makes them available to a wider testing audience. Let's do that now, so there is more time for testing and releasing RT5 this month.

[heckflosse]

Merged cppcheck and let it open for further work

[Floessie]

@heckflosse @Beep6581 When and how would you like me to do the clang-tidy bulk changes (if at all)? I could create a temporary branch from master and apply those changes only for testing. If it still works and builds (which I expect), I could do the same after the soft-proofing branch is merged. What do you think?

[Beep6581]

At this point we need all the testing we can get. If the clang-tidy changes are to be part of 5.0, and if they're in good enough shape, then it would be better to merge them into master ASAP so nightly builds become available straight away for testing. If these are the kinds of changes that could cause major problems (for example corrupt parts of images which only occur in one image here and there, making them difficult to discover and trace) then it's better to not include them in 5.0. You decide.

[Floessie]

@Beep6581

If these are the kinds of changes that could cause major problems (for example corrupt parts of images which only occur in one image here and there, making them difficult to discover and trace) then it's better to not include them in 5.0. You decide.

I've confidence in the clang-tidy checks that included in the list. Most of them are about streamlining the code, some of them simplify logic. This could be some kind of a problem, and if they are too invasive, I'll skip them. But I only had positive experience with those checks.

Okay then, I'll prepare a branch ASAP so that you can have a look on what will change and what you like or don't like about it. But if it is merged to master, be aware that it will make merging harder. That's why I suggested doing it ALAP. Basically the same rules apply to your astyle run.

HTH Flössie

[Beep6581]

@Floessie a branch would be good :+1:

if it is merged to master, be aware that it will make merging harder

@heckflosse @Hombre57 @Desmis please weigh in, as it affects the large number of outstanding patches and branches.

[heckflosse]

No problem with my outstanding work.

[Hombre57]

I'll probably merge the soft-proofing branch tonight, so there will be no issue for me but for you here.

Regarding the other opened branch :

• spot-removal-tool : I'll be able to merge your changes in this branch, there's not much collision with existing code
• gtk3 : can't say, @Beep6581 will handle any merge
• SaveStrategy : stalled branch, sadly, because too much suggestions that can't be satisfied rapidly. It was 95% done IIRC. This branch touch the GUI part mainly. I need to see you changes @Floessie, so I'll see if I can merge w/o too much trouble.
• locallab : @Desmis' branch, this one is a big and active one, despite the fact that we'll open a concurrency branch after the official switch to Gtk3 and the merge of spot-removal-tool branch (that might take some part of the code)
• newwavelet : @Desmis' branch, seem to be stalled too.
• gammadif : @Desmis' stalled branch, because not enough test from the devs. Considering the recent comment from @ellelstone, I think we should keep it alive. It will however be very difficult to update it to the current master.
• cacorrectfix : @heckflosse's branch, don't know the status of this branch.

Adam's Compress thumbnails PR is stalled too. I don't think we can hope for someone to it take over.

I'll tell you my final words about my branches after the analysis of your clang branch @Floessie .

[Beep6581]

I would really like "newwavelet" to make it into 5.1! It had some bugs the last time I tried it, but @Desmis worked on it after that and I haven't had a chance to properly test it since. "gammadif" I didn't test because I didn't understand what it really did or how to test it. After recent discussions I'm also interested in it.

[Floessie]

@Hombre57 No need to get in a hurry because of me. My plan is to provide you all with a clang-tidy branch to see what will change. This will be a temporary one, which I'll ditch later. When you all and especially @Beep6581 is okay with it, I'll create a new one after all branches for 5.0 have been merged and near the point where @Beep6581 will do his astyle run. Sounds sane?

To ease merging other branches, I could branch them off and apply clang-tidy on them, too. Merging them should then be not so much of a problem. Theoretically, git is smart enough to detect if lines are the same even if they were changed independently.

[Desmis]

For me, you do what you want, there will always a way to fix after, either alone or with the help of someone :)

newwavelet: Unless I am mistaken, Ingo has taken time to examine the code some weeks ago.

gammadif: it is an "old" branch, with more features as it seems

locallab: there are two different things in it, as I have already said. 1) the algorithm itself "U-point type", now from my point of view works correctly (it can probabaly be improved and optimzed!), with similar performance has CN2 or Nik Software, which is to say that there is virtually no need to be trimmed objects, or use masks. Even if we can use.

2) management of multiple points, which faces the problem (dcrop.cc) to place the detection area (hue, ligthness, chroma) outside the preview (actually generate crash or dysfunction). This will be the same problem whatever the "multipoint" method, due to RT pipeline.

:)

[Floessie]

Here is the demo branch. Some remarks:

• readability-braces-around-statements failed with macros. Maybe astyle can help here (uncrustify definitely can).
• readability-avoid-const-params-in-decls found nothing. :+1:
• readability-redundant-control-flow neither. :+1:
• misc-redundant-expression neither. :+1:

HTH Flössie

[Beep6581]

Hello How are we with the clang-tidy and >=gtk+-3.20 branches/patches?

[Floessie]

@Beep6581 I have to clean some things up on the clang-tidy branch (for instance dcraw.cc has to be reverted prior to merge) and have very little time this week. I can't promise to finish or recreate it this week, but I'll try to somehow make it.

[Hombre57]

I tried to get RT works with gtk>=3.20, but I'm about to give up. Still on this issue at the moment.

[Hombre57]

That's official : I give up on making RT work with Gtk3.22. I'd recommend to enforce the use of Gtk3.18, since it's the last stable version that RT can use. If you're okay, you should replace >=3.16 by =3.18 on line 229 & 232 of the main cmakefile.

Removing issue #3446 from the blocking issue.