jpeg v9b (Discuss bump) (Anyone interested?)

[DomT4]

There's been a new version of jpeg out for a while now, almost a year actually, but we haven't bumped to it yet.

The previous discussion on this was #17733. There was largely dread due to the bot not existing at that point, revisions being a longer rebuild process, and generally the need for maintainers to spend a long, long time sorting everything out in regards to testing changes and dealing with fallout. I guess at least, largely, issues 1 & 2 are negated by the existence of the wonderful bot these days.

One of the big hiccups is that jpeg v9 changes the dylib version name, and consequently, likely many if not all of the brew uses need to be revisioned to cope.

dyld: Library not loaded: /usr/local/lib/libjpeg.8.dylib
  Reason: image not found

They would also need to be tested beforehand to check whether they will actually build against the newer version. We have the previous newer version in Homebrew/Versions, but that hasn't been updated to the new .a bump.

The fact that, beyond the original creation of the jpeg9 formula, absolutely nobody has teed up a PR against it to either improve it or bump it to the .a release perhaps is telling on how much general urgency there is to move off the .8 release. In addition, the Core here doesn't seem to have taken much of a jpeg9 plz pounding, to say the least.

I guess we have the capacity to more easily bump the formula at this point, it's just whether or not there's the desire or even need to get it done.

For completeness, this is the current brew uses list. Apologies for the terrible formatting:

allegro5        gegl            imlib2          little-cms      povray36
analog          ghostscript     io          little-cms2     qemu
argyll-cms      gle         jasper          logstalgia      sane-backends
center-im       gnuplot         jp2a            mal4s           scantailor
dcraw           gource          jpeginfo        mapnik          sdl2_image
devil           gpac            jpegoptim       mapnik071       sdl_image
djvulibre       gphoto2         jpegrescan      minidlna        sfml
eet         grace           leptonica       mjpegtools      swftools
epeg            graphicsmagick      leptonica169        open-scene-graph    tiff2png
fbida           gst-plugins-good    libbpg          open-sg         ufraw
ffmpegthumbnailer   gst-plugins-good010 libgaiagraphics     openslide       vice
flactag         gst-plugins-ugly    libgeotiff      pdf2htmlex      vncsnapshot
fltk            gst-plugins-ugly010 libpano         php53           webp
fontforge       gtk+            libquicktime        php54           wine
fox         gtk+3           libsvg          php55           wxmac
freeswitch      hdf4            libtiff         php56           x11vnc
gd          htmldoc         libuvc          podofo          xplanet
gdal            imagemagick     libwmf          poppler         yarp
gdk-pixbuf      imagemagick-ruby186 links           povray          zbar

Short and sweet: Do we want to do this here at this point, or should we just bump the Homebrew/Versions one indefinitely until demand catches up with jpeg9?

No rush on this discussion in the slightest. It's been on my radar for a while, but I have no strong feelings beyond a willingness to help get it done.

[jacknagel]

Looking at http://www.ijg.org/files/, the trend seems to be January releases, so it will probably be easier to wait for the next version.

[DomT4]

Sounds good to me. No need to put a rush on this at all, and waiting for the presumably upcoming .b release makes sense.

[vszakats]

No release in or since January 2015. Shall we keep waiting or move along with the update? Currently hosted version is more than 3 years old.

Development is continued on this website: http://infai.org/jpeg/ Above page details the changes implemented in version 9 and 9a.

We're building this project as part of another one also on OS X and there hasn't been any build fallout. As for any API incompatibilities, nothing suspicious can be found in the change listing.

[DomT4]

As for any API incompatibilities, nothing suspicious can be found in the change listing.

The dylib version number changed, so everything that links into it would need bumping regardless of API compatibility.

[vszakats]

Yes, I understood that. Anyhow, sooner or later it has to be done anyway.

[DomT4]

I'm cool with it, but get Jack's :+1: as well.

[tdsmith]

It would be nice to coordinate this with -science.

[DomT4]

Yeah, Science have 18 formulae with jpeg linkage (optional/recommended/mandatory), Versions has 11, X11 has 5, PHP has 1, Python has 2.

@iml recently handled the GCC-5 bumps for Science, so his views on this would be cool.

[ianml]

I would have liked the bot time limits relaxed if we were to do another series of GCC/GFortran bumps, but jpeg looks much easier. Of the 18 at least one is statically linked and some others are not bottleable.

[MikeMcQuaid]

@iml Rather than bumping the limits it's probably better to split it into multiple PRs.

[DomT4]

@vszakats If you want to start the ball rolling by teeing up a PR for Homebrew/homebrew that'd be welcome. Nobody has objected yet to the idea, but a PR will give us a better base to work from in regards to knowing how many or few problems we'll encounter.

[vszakats]

By now I'm a little confused on what to split/merge into single/multiple commits/PRs. Had to throw away some hours of work a few days ago because of apparent misunderstandings when trying to split a complex-ish update into multiple passes (PRs). To avoid losing work again, what would be a good strategy to please the bot, maintainers and also let myself follow all the moving parts?

Does this look reasonable?:

• PR #1
  • commit #1: jpeg formula
  • commit #2: first 10 dependents
• PR #2
  • commit #1: next 10 dependents

[DomT4]

Since we're updating the main formula and all dependants will need a revision it'll need to be done in one big PR. We can bump the timeout if it proves necessary after the first attempt. You'd just do it like this:

• Create a PR
  • Update jpeg => Commit 1
  • Update first brew uses jpeg => Commit 2
  • Update second brew uses jpeg => Commit 3

And so on. The list for the core is:

allegro
analog
argyll-cms
center-im
dcraw
devil
djvulibre
eet
efl
epeg
fbida
ffmpegthumbnailer
flactag
fltk
fontforge
freeswitch
gd
gdal
gdk-pixbuf
gegl
ghostscript
gnuplot
gource
gpac
gphoto2
graphicsmagick
gst-plugins-good
gst-plugins-ugly
htmldoc
imagemagick
imageworsener
imlib2
io
jasper
jp2a
jpeginfo
jpegoptim
jpegrescan
leptonica
libbpg
libgaiagraphics
libgeotiff
libgxps
libpano
libquicktime
libsvg
libtiff
libuvc
libwmf
links
little-cms
little-cms2
logstalgia
mal4s
mapnik
minidlna
mjpegtools
open-scene-graph
openslide
pdf2htmlex
podofo
poppler
povray
qemu
sane-backends
sdl2_image
sdl_image
sfml
swftools
tiff2png
ufraw
vice
vncsnapshot
webp
wine
wxmac
x11vnc
xplanet
zbar

Not all of those may be required. You don't need to bump :optional usage.

[tdsmith]

To split this into multiple PR's in such a way that there's not a window where the bottles for jpeg dependents are broken, you'd have to include the jpeg change in each PR, or else all the dependents will be bottled wrong.

I think that's actually a reasonable thing to do, though it'll take some fussing with git to actually apply the PRs when we're ready to push them.

[vszakats]

Thanks @DomT4. Here goes the fun: https://github.com/vszakats/homebrew/commits/jpegupd

I took this bump opportunity to make minor formula updates, in case they are trivial and isolated.

Will continue tomorrow.

[ used this command to filter dependent projects: grep depends_on *.rb | grep [\"\']jpeg[\"\'] | grep -v :optional ]

[DomT4]

I took this bump opportunity to make minor formula updates, in case they are trivial and isolated.

The bot will test everything from scratch, so feel free to fix up anything highlighted by brew audit --strict that isn't a description. Not mandatory for existing formulae obviously, but as much as you want to do is great, thanks :star2:.

[tdsmith]

Not to contradict Dominyk but it's more important to me that these PRs be easy to review and merge than that we eliminate every last single quote.

One formula per commit, one commit per formula would be nice, too.

[tdsmith]

The kinds of changes you made in that branch look fine to me, @vszakats.

[DomT4]

Tim's views "outrank" mine. Ignore me and follow his advice if we ever clash :smiley_cat:

[vszakats]

I'd like to stick with the same class of trivial updates, but I'm not promising one formula per commit. If that's a stopper, I'd rather stop at this point.

[MikeMcQuaid]

@vszakats Why not, out of interest? If it's a pain I suggest using something like GitX or git gui which makes it a bit better. It is how we expect these things to be, I'm afraid.

[vszakats]

It's 10 times the busywork and 10 times more opportunity to mess up (regardless of the UI - I'm ok with command-line). It's also 10 times more work to undo/revert in case anything goes wrong. If the trivial updates are the main concern I can stop with those, though I'd feel it to be another waste of opportunity and time.

[MikeMcQuaid]

@vszakats It makes life significantly easier for maintainers to see what's changed and to revert individual commits. I'm sorry you feel it's a waste of your time. I've tried to be pretty easy on your pushback on this and various other issues (e.g. fixing brew audit --strict without adding tests) but I think you've been contributing to Homebrew long enough now that your contributions should be treated like anyone else's (e.g. one formula per commit).

[vszakats]

We feel differently, and it's okay. It'd be over my spare time limits to follow an all or nothing approach for these 70 formula updates, so I'm respectfully pulling out from this one.

[MikeMcQuaid]

Ok, sorry to hear that.

[L2G]

It looks like no one else is taking a stab at this, so... anyone mind taking a look at this branch to see if I'm on the right track?

https://github.com/Homebrew/homebrew/compare/master...L2G:jpeg-9a-migration

[L2G]

Thanks, @tdsmith. (He responded to me on IRC with "Yes, it looks fine :+1:")

[DomT4]

Yeah, looks good. You'll need to give @Homebrew/science a heads up as well, I think they pretty heavily use jpeg and any sudden change is going to bite them.

The last time we changed something so cross-tap we teed up PRs with revisions for everything necessary from each tap, merged the core change, and then tested PR from tap 1 & merged it, tested PR from tap 2 & merged it, and so on.

As far as I know, Versions, Science, Python, Games & X11 all have jpeg-using formulae.

[L2G]

Yup, I plan to line up pull requests for all affected homebrew/* taps.

[sjackman]

Great. Thanks, Larry. The Homebrew-science formula that directly depend on jpeg are astrometry-net blast enblend-enfuse insighttoolkit lammps mathgl openalpr opencv opencv3 openimageio paraview r stiff sumo vigra vips vips7 visp vtk vtk5

[vszakats]

[FYI] 9b is out: http://www.infai.org/jpeg/ http://ijg.org

[DomT4]

I'm still happy for someone to pick this update up and get it done.

[DomT4]

This is still open for anyone to run with, but closing here.