A new beta

[John-Colvin]

I think that because of https://github.com/ldc-developers/ldc/issues/1819, https://github.com/ldc-developers/ldc/releases/tag/v1.1.0-beta3 is getting less testing than it would otherwise. The sooner beta4 comes along the better, even if it's a very small change from beta3

[redstar]

I try to create a new beta this weekend.

[kinke]

Any new plans? I'd say let's try to put out 1.1 in 2016; ideally one more beta/RC is enough. I'd like to see #1869, #1861, #1845, #1840, ideally #1856 and possibly even #1832 in there.

@klickverbot: It somehow feels as if the spurious OSX Travis issue doesn't happen that often anymore. Do you think we can include your OSX commit at least in the next beta/RC?

[dnadlinger]

Do you think we can include your OSX commit at least in the next beta/RC?

I'd still wait until the next release if we are uncertain about what is going on (which I believe we are). There is no reason to delay the release any longer by trying to fit in more than necessary from master.

[kinke]

I'd like to see at least all of current master, except for OSX then (+ unreverting the runtime function type checks), in the 1.1 release. The 57-2 commits (behind master) are mostly fixes (incl. important ones like #1819, #1822, ARM soft-float, debuginfos, and half-fixed #1829) and also affect the command-line (incl. ir2obj cache renaming). Nothing particularly dangerous or insufficiently tested but all well worth to be included IMO.

[JohanEngelen]

Shall we start to cherry-pick stuff onto the 1.1.0 release branch?

[kinke]

As long as there's a sufficient consensus wrt. what should make it into 1.1, I'd rather merge master into release-1.1.0 (read: rebasing) and revert the 2 commits, to have a cleaner history and ease branch comparisons.

[JohanEngelen]

Not a bad idea. I have no preference.

[kinke]

As @redstar seems too busy at the moment, shall the rest of the team prepare a new beta? E.g., me doing the Windows builds, Johan the OSX ones and David the Linux ones? I'd need to edit the release scripts and may use other versions of tools than Kai, so there'd surely be some slight differences, but I'd like to get a new beta out soon.

[PetarKirov]

Please do so! A new release would be huge improvement on the status quo, considering that beta-3 broke the build of almost every dub package (e.g. even dub --compiler=ldmd doesn't work with vibe.d). See also #1890.

[dnadlinger]

@kinke: Sure – I had documented my release process at https://wiki.dlang.org/How_to_release_LDC, but Kai might have been building the packages differently, of course. I usually used an EC2 instance to build the Linux packages out of convenience, but a VM with a suitably old Linux version will do as well. I think we are now shipping statically linked binaries anyway, though, so the version requirement might not be as stringent anymore.

[JohanEngelen]

@kinke I've been already doing the OSX builds recently, so definitely. Hope we can have a new beta very soon.

[kinke]

Alright, so I'd suggest #1889 (minus 4 Travis OSX jobs) + #1891 + LTO backport for the next beta. I'd prefer calling it a release candidate to emphasize that we're pushing for the final release and that the plan is to check for any remaining major issues only. The final could then be rolled out as christmas present. ;)

What do we do about the ARM build? Postpone it or ask @smolt and/or @joakim-noah?

Preparing the release over the course of this week would be fine by me. @klickverbot: Would you be willing to take care of the official anouncements (and the Linux x86 builds - I only have a Ubuntu 16.04 VM)? I can prepare a summary of all closed issues since beta3 in the meantime, and Johan would be welcome to shortly summarize his feature additions like LTO for the release notes.

[JohanEngelen]

Cool! Just call it "beta4", let's not complicate things further ;-) As soon as you have the release draft up, I'll add LTO stuff. (in future, I think we should open a release draft as soon as development begins. Easier to add (and not forget) things on-the-go than retro-actively)

[redstar]

Sorry for the inconvenience right now! I currently have some trouble which prevents all work on LDC. :-(

@klickverbot I used the scripts for building. I use Ubuntu 12 TLS in a VirtualBox machine for building (both 32 and 64 bit). I added gcc 4.8.1 backport to the standard Ubuntu and a self-compiled patchelf. For the Windows build I use the msbuild scripts from the repository.

@kinke Good proposal. I found it always annoying to scan the issue and commit history to find the relevant entries.

Thanks again!

[kinke]

No worries Kai, take all the time you need and 'halt die Ohren steif'. ;)

Just call it "beta4", let's not complicate things further

Alright.

[JohanEngelen]

@redstar Thanks for chiming in! :-)

[joakim-noah]

Regarding ARM, I don't have a linux/armhf board available, which is what Kai was using for that build. I guess we can delay that build.

[kinke]

I tweaked the MSBuild project in ldc-scripts a bit and am now able to build the 2 Windows releases. Note that I updated dub to v1.1 and made sure LDC is compiled by itself.

[dnadlinger]

@kinke:

Would you be willing to take care of the official anouncements (and the Linux x86 builds - I only have a Ubuntu 16.04 VM)? I can prepare a summary of all closed issues since beta3 in the meantime, and Johan would be welcome to shortly summarize his feature additions like LTO for the release notes.

Sure. I'm a bit short on time right now, though (end of term and thesis project approaching), so it would be great if you could indeed prepare a summary of the changes. Thanks!

[kinke]

@klickverbot: Great, thanks. If you agree with the proposed tag, please go ahead and merge #1889 and #1891.

[kinke]

I've prepared the release draft description, using filters like is:issue closed:>2016-10-08 to browse through the closed issues and merged PRs since beta 3 was released. It's definitely time to get 1.1 out soon, the change log is getting huge. ^^ Well done, guys! 👍

[JohanEngelen]

The draft looks great @kinke , excellent work!

[kinke]

@JohanEngelen: Thanks ;) - do you have time to backport the LTO fix into release-1.1.0 now that David has merged #1889? Otherwise I could give it a try too.

[JohanEngelen]

ok

[kinke]

I actually think we should ship brand-new dub 1.1.1-beta1, as it fixes at least one important issue/regression (e.g., vibe.d on Unixoids - https://github.com/dlang/dub/issues/990) and not-so-irrelevant issue https://github.com/dlang/dub/issues/931: https://github.com/dlang/dub/compare/v1.1.0...v1.1.1-beta.1

[kinke]

I've noticed that merging #1889 into release-1.1.0 screwed up the history. So I took the PR branch, cherry-picked the later commits to origin/release-1.1.0, made sure there were no file diffs with origin/release-1.1.0, then replaced my local release-1.1.0 with the fixed-up branch, additionally merged origin/master into it and then force-pushed it. The result is that it's 9 commits ahead of (and 0 behind) master (merges, reverts/backport and some redundant commits - testing LLVM 3.9 for Travis OSX, IR test fix, LLVM 4.0 fix).

[JohanEngelen]

@kinke Ready to tag beta4 then!

From now on, we should no longer merge master, and just cherry-pick onto release-1.1.0. It is much easier to work with a cherry-picked history than merges when bugs emerge.

[kinke]

I'm waiting for a green AppVeyor, will then probably get rid of the 3 redundant commits with an additional force-push and then tag. [I'm already building temptative Windows releases right now. ;)]

I don't agree that there should only be cherry-picks from now on. That is only necessary if something makes it into master that we don't want in 1.1 before 1.1 is actually released. The right time for such a 'freeze' is when 1.1 final is released. Until then, I propose to delay merging risky PRs into master. This way, we don't end up with a parallel stream of bugfixes, improvements and small new features, which makes branch comparisons very cumbersome.

[JohanEngelen]

My experience with non-cherrypicking release schedules has not been good. This 1.1.0 release cycle is an example of it, but I have more experience with this mechanism during my years of Inkscape development. Not cherry picking means that all development is stalled until a release happens; if not, the release keeps getting bigger with new bugs popping up, history becomes harder to track down/bisect because of relatively large merges with merge-conflict amends, "can this please make it into the release?"-hopes are increased but delay the release. If, for whatever reason, the release doesn't happen as quickly as hoped, bad things build up: bigger and bigger unmerged feature PRs, merge conflicts when PRs can finally be merged, forgotten branches, demotivation/loss of interest.

The rules of a cherry-picked release cycle are easier to understand, easier to deal with the history, and I think will be easier to manage with several people at once. It is not a "freeze": cherry-picking is easy, easier than merging, and mistakes are easier to spot. (It's the opposite of a "freeze"; the whole time we've been "frozen out of" updating to 2.072!)

[kinke]

I simply did a reset to master and cherry-picked the 2 reverts and the backport - 3 commits ahead of master, as simple as that. I think seldom force-pushes are okay as long as the release isn't official.

the whole time we've been "frozen out of" updating to 2.072!

For early birds, a temporary v2.072 branch could be used, just like we did when transitioning to the DDMD front-end. That worked out quite well, did it not?

Edit: tagged. Edit2: from my perspective, the release cycle only begins with the final release (branching off master at that time, possibly already with a stack of release-specific commits such as the OSX revert here). Alphas and betas etc. are just experimental previews (which don't have any continued support like finalized releases). When going with a release-1.1.0 branched off master at the time the first beta was released, we'd already have ~100 diverging commits from master before the final release, for basically exactly the same except for OSX shared libs. As soon as the release is finalized, there should be cherry-picks only, I agree with you there, for the reasons you mentioned. But normally a release shouldn't take as long as 1.1, I think that was really an exception.

[JohanEngelen]

a temporary v2.072 branch could be used

I didn't create it yet. Because it is a drain on energy to keep a separate branch alive for a long time (dealing with merges, for example).

For me, the betas are not very useful if we keep adding functionality to them (I've been guilty of this too). Regardless, beta4 is out, building mac binaries now!

[dnadlinger]

Building Linux binaries.

[dnadlinger]

We'll also have to figure out a way to build the Linux/ARM and FreeBSD binaries. Situations like these are exactly why everything used to be documented at https://wiki.dlang.org/How_to_release_LDC.

[dnadlinger]

Seems like the prerequisites (newer GCC, …) are also missing from https://github.com/ldc-developers/ldc-scripts/blob/master/ldc2-packaging/0-ubuntu-setup.sh.

[dnadlinger]

So we are shipping DUB 1.1.1-beta.1? Edit: Yes, we are.

[dnadlinger]

Apparently, the scripts don't actually build the package correctly anymore – libconfig.so.8 ends up not being marked executable, and the exe patching is broken. I'll revert that back to properly using linker flags.

@kinke, @JohanEngelen: This might require a change to the CMake scripts. I suppose since this is only a beta, I'll just update the tags rather than skipping to -beta5, which would be the "proper" thing to do.

[dnadlinger]

Somebody shoot me:

extra_flags+=("-DCMAKE_EXE_LINKER_FLAGS='-static-libstdc++ -Wl,-rpath,\\\\\$\$ORIGIN'")

Hopefully this works now. The quoting is of course specific to the Makefile generator, but that's what the install scripts use anyway. We should properly fix the quoting in the custom linker commands, though.

[dnadlinger]

I'll announce the beta tomorrow; it's way too late right now.

[kinke]

I tested the Linux x64 package on Ubuntu 16.04 by building a vibe.d project template (with both LDC and LDMD) - works fine, except that https://github.com/rejectedsoftware/vibe.d/issues/1611 isn't fixed by https://github.com/dlang/dub/issues/990, so the manual OpenSSL dependency is still required. :/ Edit: fixed when removing the ~/.dub cache.

[JohanEngelen]

@klickverbot : Important beta4 issue: https://github.com/ldc-developers/ldc/issues/1897 fixing it now. (https://github.com/ldc-developers/ldc/pull/1898)

[dnadlinger]

Beta5 it is, then.

[dnadlinger]

Tagged beta 5; building Linux packages.

[kinke]

Rebuilding the Windows packages.

[kinke]

Ready.

[dnadlinger]

Announcement in ~6h unless any other last-minute issues due to the changed build process pop up.

[JohanEngelen]

https://github.com/ldc-developers/ldc/issues/1901

[dnadlinger]

I've held off on the announcements since even without, a couple of issues have already popped up.

[kinke]

Beta 6 tagged, drafted and Windows packages being built.

[JohanEngelen]

I've uploaded the OS X package.

[dnadlinger]

Linux packages will be up in a couple of hours.