Discuss: changes to Stack release process for Linux

[borsboom]

I'd like to discuss my current thinking on some changes changes to simplify and streamline the Stack release process for Linux:

• Once confirmed that they work for everyone, standardize on statically linked Linux binaries that work on all distros (no worries about which version of libgmp is installed). This will simplify the installation instructions and build process.
• Drop official 32-bit Linux support. While Stack will continue to work on 32-bit platforms and be able to download 32-bit GHC bindists, we will stop providing 32-bit binaries. It's becoming increasingly clear that 32-bit x86 is no longer an important platform, and it's getting difficult to make build environments for 32-bit (for example, Fedora has stopped providing 32-bit ISOs). [UPDATE: received feedback that 32-bit binaries are still very much being used, so will keep building them for now]
• Stop providing .debs and .rpms [EDIT: for new Linux distribution versions] and remove our APT and YUM repositories. Installation standardizes on get.haskellstack.org and upgrading is done using stack upgrade (once that command can just download a binary instead of building Stack from source). Note: for Fedora there is still the Copr repo maintained by @juhp, and ideally Stack will be added to the official Debian/Ubuntu/Fedora/Red Hat repos at some point [UPDATE: based on feedback, will keep uploading new Stack packages for currently existing Linux distribution versions. There are also now official haskell-stack packages for Debian and Ubuntu].
• Update get.haskellstack.org to no longer use the APT/YUM repos, and to install stack in ~/.local/bin instead of /usr/local/bin. This also will make it possible to no longer require sudo (although in that case users may have to manually install some dependencies like make).

To get there, we'd follow a phased process:

v1.2.0:

• [X] start releasing 64-bit static Linux binaries for people to try
• [X] announce these plans

v1.3.0:

• [X] implement binary code path for stack upgrade (#1238)
• [X] get.haskellstack.org no longer installs from APT/YUM repos, and installs stack in ~/.local/bin instead of /usr/local/bin
• [x] remove references to our APT/YUM repos from install documentation
• [ ] stop updating APT/YUM repos
• [X] announce these changes

Next version after v1.3.0:

• [ ] stop providing 32-bit and dynamically linked Linux binaries [UPDATE: continue providing 32-bit dynamically linked Linux binaries until we are able to build 32-bit static Linux binaries]
• [ ] remove the APT/YUM repos

[UnkindPartition]

I think dropping support for x86 is very premature. It may be an unimportant platform, but people who are stuck with it are (hopefully) not unimportant.

For instance, until a year ago, I was using a 32-bit Dell Vostro. I am sure there are plenty of people around who haven't bothered, or cannot afford, to transition to x86-64.

Do you really want to deny them access to this crucial part of the Haskell infrastructure?

As for the build environments, I'm sure Debian will keep supporting x86 for a while (along with much more obscure architectures).

[Blaisorblade]

for example, Fedora has stopped providing 32-bit ISOs

Fedora's job is to actively break stuff people need . OK, not really, but that doesn't show a platform is unsupported, and Ubuntu still lists 32bit on their second download page. http://www.ubuntu.com/download/alternative-downloads

And Linux users are especially diverse, and tend to stick to old computers, and some use 32bit Linux on 64bit hardware to this day. http://www.phoronix.com/scan.php?page=article&item=ubuntu-1604-3264&num=1

it's getting difficult to make build environments for 32-bit

Do we need community contributions for the effort? It's somewhat common to delegate part of the support job for uncommon platforms to the community. And the job should be maintaining the scripts to setup a virtual machine for the build, so I'd hope that's limited.

[borsboom]

Community contributions would be great. The biggest thing that would help is having a version of GHC that works on 32-bit Alpine Linux, since Alpine is what we're using to build the static stack binaries and currently there are no 32-bit GHC builds that we are aware of. For 64-bit, we're using the packages produced by https://github.com/mitchty/alpine-linux-ghc-bootstrap. In addition to that, 32-bit GHC bindists built on Fedora 24 (with libtinfo.so.6) and Arch Linux (with libncursessw.so.6) would be most helpful.

[juhp]

No strong objections - though personally I am also cautious about dropping 32bit Linux. While it is true that Fedora is de-emphasizing the i686 arch, it is still a primary arch for the coming Fedora 25 release, and 32bit install media is available for Fedora Workstation.

I plan to continue building 32bit stack for Fedora anyway in my Copr repo as long as it is a Fedora primary arch.

[dag]

for example, Fedora has stopped providing 32-bit ISOs

Huh? The download page lists 32-bit images both for the live and net installer.

[Blaisorblade]

Uh, that's still true for the Fedora prerelease, good: https://getfedora.org/en/workstation/prerelease/

Meanwhile, I've asked for community contributions on Twitter, since this helped for Travis. Let's see what happens. Any retweets appreciated: https://twitter.com/Blaisorblade/status/773258292994932740

[borsboom]

Feedback heard: I'll keep the existing 32-bit binaries going for now. I'd really love to get static 32-bit binary working, but we need a 32-bit GHC that works on Alpine Linux for that (as mentioned above).

[kadoban]

Dropping the .deb files (and .rpm for others I guess) is pretty unfortunate. There's way too many programs on my machine for me to update each manually, I'm certain to fall behind in versions.

[borsboom]

@kadoban: We're hoping that others will pick up the task of running repos (as @juhp as done for Fedora) or getting the packages into the official repos. The current way we're doing it is a hack that is becoming increasingly unmaintainable; we're just using FPM rather than following proper procedures for building packages.

We're also going to improve the upgrade path in stack itself by having it warn if you're running an out-of-date version and having stack upgrade download a binary rather than build from source if you installed from an official binary originally.

[cblp]

In which aspect may stack upgrade be better than system package manager?

[dpwiz]

we're just using FPM rather than following proper procedures for building packages.

So, it's an issue of having proper build tools then?

[Blaisorblade]

stack upgrade works for all users, with root privileges or not, and current stack contributors can actually maintain support for stack upgrade, while we can't provide good packages for all distributions. @borsboom is there a link to the FPM-based scripts? I've only found https://github.com/jordansissel/fpm.

[borsboom]

So, it's an issue of having proper build tools then?

@wiz: it's not just that, but the overhead of maintaining them as new distros and versions of distros are released and running them for every Stack release. It makes more sense for people who have a real knowledge and interest in those distros to maintain packages for them, rather than trying to centralize it among the Stack maintainers.

[borsboom]

is there a link to the FPM-based scripts? I've only found https://github.com/jordansissel/fpm.

@Blaisorblade: fpm is used in https://github.com/commercialhaskell/stack/blob/master/etc/scripts/release.hs. The various Vagrantfiles in https://github.com/commercialhaskell/stack/tree/master/etc/vagrant need to have the right environment to build the packages for each distro as well.

[nomeata]

Stack will be added to the official Debian/Fedora/Ubuntu/Red Hat repos at some point.

Has been in Debian (and hence Ubuntu) there since a while: https://packages.debian.org/sid/haskell-stack We might be a bit slow in upgrading, as we follow the Stackage LTS releases.

[nh2]

Hmm, I do like the APT repository a lot because it integrates stack in my usual update practice, and the way of guaranteeing the integrity of the updates (apt-key) is very well understood. If providing APT was zero-cost, I'd immediately vote for that.

At the same time, I care about stack maintainers' time as well. @borsboom, which part of maintaining these repositories is the pain point? Would it be much simpler to e.g. provide an APT repository that simply contains the static binary packaged up?

[m4lvin]

About non-sudo installations: What if my {distribution, admin, ...} gave me an old version and I want to have a newer stack in ~/.local/bin? Do I get that by running "stack upgrade"?

[Blaisorblade]

Yes (if your old version is new enough). And with or without an existing stack, you can just download a new one into ~/.local/bin and it'll be preferred, as long as ~/.local/bin comes in PATH before the other location where stack is installed.

[charles-cooper]

Based on feedback at https://www.reddit.com/r/haskell/comments/5i9yv1/release_v130_commercialhaskellstack_github/ I would recommend we reconsider removing package manager support (apt/yum), since this seems to be a common expectation in the linux community.

I think at least we should have a plan for passing the baton of maintenance responsibility to another individual or group.

[borsboom]

What we'd really prefer is for people who are good at this sort of thing to add Stack to the distributions' official repos. At this point Stack is well enough established that I don't think any distributions would turn it down. Ubuntu does already include a haskell-stack package, so that's a big chunk of Linux users right there. It's not bleeding-edge, but you're just a stack upgrade away from the latest version (easier since 1.3.0, since stack upgrade will just download a binary if it can instead of building from source).

We'd be perfectly happy for third-parties to start maintaining their own repos as well, and happy to link to those from the installation instructions. The code that builds the packages is all in the repo (see the maintainer guide), although I'd suggest not using it because it's really hacky (we use FPM instead of properly creating the packages).

[charles-cooper]

It makes more sense for people who have a real knowledge and interest in those distros to maintain packages for them, rather than trying to centralize it among the Stack maintainers.

This makes perfect sense. I hear you. It makes sense that the stack maintainers should focus on maintaining stack and the distro maintainers should focus on maintaining their distros. The only thing is I think we should at least have a continuity plan for guaranteeing the community continued access via the official package managers instead of just dropping that support and kind of crossing our fingers that someone will fill in the vacuum.

[m4lvin]

There is a debian package too https://tracker.debian.org/pkg/haskell-stack :smile:

[borsboom]

So now we have Ubuntu and Debian covered in their official repos. Fedora is covered by @juhp's copr repo. That really leaves only CentOS.

Here's what I'm thinking: I'll continue to upload new Stack packages to our existing repos for currently existing Linux distribution versions since that adds very little overhead for now and avoids breaking people's existing scripts. However, I won't add repos for any new distro versions, and as old distro versions become unsupported by their vendors they will drop off the list as well.

[mkoloberdin]

Arch Linux: https://www.archlinux.org/packages/?q=haskell-stack

[varosi]

Do Haskell stack on Debian has a ARM builds, too?

2016-12-16 21:36 GMT+02:00 mkoloberdin notifications@github.com:

Arch Linux: https://www.archlinux.org/packages/?q=haskell-stack

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2534#issuecomment-267677801, or mute the thread https://github.com/notifications/unsubscribe-auth/AFxXhuPKOniYY3Y5R3YVhYdP1N75sZ0Eks5rIugggaJpZM4JsP1o .

[borsboom]

@varosi According to this page it appears to. Not really relevant for this issue, since we were never creating .debs for ARM anyway, though.

[varosi]

Will you in the future?

[borsboom]

No, since we're planning to stop creating any .debs at all in the future, for any architecture.

[varosi]

Okay, thanks!

2016-12-19 0:51 GMT+02:00 Emanuel Borsboom notifications@github.com:

No, since we're planning to stop creating any .debs at all in the future, for any architecture.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2534#issuecomment-267853059, or mute the thread https://github.com/notifications/unsubscribe-auth/AFxXhu4oPavMvDm0j-BEZfxEs25C8sS0ks5rJbjqgaJpZM4JsP1o .

[bwbaugh]

Installing the haskell-stack package on Ubuntu 16.04, stack upgrade failed until I installed zlib1g-dev. Would it be worth making the haskell-stack package depend on it? On the other hand, keeping the dependencies as minimal as possible might be preferred.

    Configuring digest-0.0.1.2...
    setup-Simple-Cabal-1.22.5.0-ghc-7.10.3: Missing dependency on a foreign
    library:
    * Missing (or bad) header file: zlib.h
    * Missing C library: z

[Blaisorblade]

On 26 December 2016 at 01:19, Wesley Baugh notifications@github.com wrote:

Installing the haskell-stack package on Ubuntu 16.04, stack upgrade failed until I installed zlib1g-dev. Would it be worth making the haskell-stack package depend on it? On the other hand, keeping the dependencies as minimal as possible might be preferred.

Thanks for bringing this up. I'd vote to add some sort of dependency since haskell-stack is a development tool anyway (and zlib is lightweight). But Debian packages allow for "optional" dependencies (IIRC, suggests and recommends) which might be suitable here (depending on Debian policies I guess), so that users can avoid zlib1g-dev if they want. I'm sure reasonable people might disagree on this, but in the end this is up to the packager (@nomeata).

[borsboom]

I'd lean toward making it required, but recommended would be OK too (since those are installed by default, unless a user specifies apt-get install --no-install-recommends).

[borsboom]

Actually... I take that back. Starting with stack-1.3.0, stack upgrade will no longer build from source, so zlib1g-dev won't be necessary anymore (somehow I misread the earlier post as being about the -dev version, and though it was only talking about the shared library).

[dbaynard]

@borsboom — with #4088 this should be updated. Can it be closed, too?

[borsboom]

Yeah, we've now done all of this, so closing.

I think it might be time to consider removing the YUM/APT repos as well, since it's been quite some time since anything new has been added to them (last version to be added to any was stack-1.5.1, and we're now on stack-1.9.3). But I'll make a separate issue/announcement about that.