search

NixOS / cabal2nix

Generate Nix build instructions from a Cabal file
https://haskell4nix.readthedocs.io
Other
361 stars 156 forks source link

cabal2nix fails on .cabal files that contain Unicode characters #408

Closed nomeata closed 5 years ago

nomeata commented 5 years ago

it seems that #334 came back, or something else is amiss:

~/projekte/tip-toi-reveng $ nix-env -i cabal2nix
replacing old 'cabal2nix-2.14.1'
installing 'cabal2nix-2.14.1'
~/projekte/tip-toi-reveng $ grep Ravensburger *.cabal
description:         The Ravensburger Tiptoi® pen is programmed via special
~/projekte/tip-toi-reveng $ cabal2nix . > ./nix/project.nix
cabal2nix: <stdout>: commitBuffer: invalid argument (invalid character)
~/projekte/tip-toi-reveng $ echo $LANG
de_DE.utf8
~/projekte/tip-toi-reveng $ LANG=C cabal2nix . > ./nix/project.nix
cabal2nix: <stdout>: commitBuffer: invalid argument (invalid character)
~/projekte/tip-toi-reveng $ LANG=en_US.utf8 cabal2nix . > ./nix/project.nix
cabal2nix: <stdout>: commitBuffer: invalid argument (invalid character)
nomeata commented 5 years ago

Ah, may not be an issue with cabal2nix, but rather with building it using nixpkg, because it works fine when building the tool with cabal directly:

~ $ cabal new-install --symlink-bindir=$HOME/.bin --overwrite-policy=always cabal2nix -j8
…
~/projekte/tip-toi-reveng $ ~/.bin/cabal2nix --version
cabal2nix 2.14.1
~/projekte/tip-toi-reveng $ ~/.bin/cabal2nix . > ./nix/project.nix
~/projekte/tip-toi-reveng $
peti commented 5 years ago

Are you using Nix on a different host OS or are you running NixOS?

nomeata commented 5 years ago

Running nix on Debian unstable.

peti commented 5 years ago

Have you configured the environment variable $LOCALE_ARCHIVE to point to your system's locale archive? Nix's glibc needs that information to find those and without it, locale support does not work.

I've experimented briefly in a Debian docker image and I believe export LOCALE_ARCHIVE=/usr/lib/locale is the right setting.

nomeata commented 5 years ago

Plausible, but doesn’t quite cut it yet:

~/projekte/tip-toi-reveng $ export LOCALE_ARCHIVE=/usr/lib/locale
~/projekte/tip-toi-reveng $ cabal2nix . > ./nix/project.nix
cabal2nix: <stdout>: commitBuffer: invalid argument (invalid character)

Maybe the locale data is incompatible?

But since this is a problem with nix, and not cabal2nix, I guess I should close this and raise it with upstream?

peti commented 5 years ago

Maybe the locale data is incompatible?

That is possible. glibc changed its locale format recently and did cause a lot of fuss in the process. Let's see ... could you try

$ export LOCALE_ARCHIVE="$(nix-build --no-out-link "<nixpkgs>" -A glibcLocales)/lib/locale/locale-archive"

instead? That should give you the locale from Nix, which should match the version of glibc for sure.

But since this is a problem with nix, and not cabal2nix, I guess I should close this and raise it with upstream?

I don't think that the Nix bug tracker is more appropriate than this one. This is not a bug in Nix, per se. My guess is that it's an issue with the way your shell environment is configured. Let's try to solve this issue here.

nomeata commented 5 years ago

Yes, this works:

~/projekte/tip-toi-reveng $ export LOCALE_ARCHIVE="$(nix-build --no-out-link "<nixpkgs>" -A glibcLocales)/lib/locale/locale-archive"
~/projekte/tip-toi-reveng $ echo $LOCALE_ARCHIVE 
/nix/store/rdzldh4fiisgjy2hfvs1bf48gprbqlkh-glibc-locales-2.27/lib/locale/locale-archive
~/projekte/tip-toi-reveng $ cabal2nix . > ./nix/project.nix
~/projekte/tip-toi-reveng $
peti commented 5 years ago

Very good. Now you can add that bit into your ~/.profile or whatever other place you feel is appropriate to make sure all your Nix-built programs work correctly. You can use the variable $LOCALE_ARCHIVE_2_27, if you like, to make sure that only appropriate versions of glibc try to use those locales. Your system might have an older version and then it would break if you point it to the more recent versions. Check out https://github.com/NixOS/nixpkgs/issues/38991 for further details, if you are interested.

The long-term solution is to make cabal2nix decode utf-8 properly without depending on the system locale. That's a long standing issue ... I've just never gotten around to fixing it. I may take a shot at this tomorrow.

nomeata commented 5 years ago

The long-term solution is to make cabal2nix decode utf-8 properly without depending on the system locale.

I think the long-term solution would be that nix-built programs use a nix-provided locale set, without requiring manual user interaction – after all, locale support is more than just file encoding. But this is getting off topic… thanks for you help!

peti commented 5 years ago

It is not that simple, unfortunately. We don't want the glibc derivation to depend on the locales. If it did, then updating the locale data would trigger a complete rebuild of the entire system. It would also be impossible to install certain locales selectively without rebuilding the entire system from scratch.