ghost commented 4 years ago

Choosenim v0.6.0: Toolchain extraction results in corruption of vital `Nim v1.2.0` files

Note: This is my first issue report ever so, don't get too harsh with me.

System details

Arch Linux x86_64
Choosenim v0.6.0 previous v0.4.0
Initial nim toolchain v1.0.6

Reproduce the problem

I updated choosenim using:

$ choosenim update self

I updated nim toolchain with:

$ choosenim update stable

Extraction took forever, almost 20 minutes (usually less than 2 minutes) with intensive cpu usage. Then I created a project with nimble with no problem and then nimble build returned this.

     Error: Could not read package info file in /home/user/projnim/projnim.nimble;
        ...   Reading as ini file failed with: 
        ...     Invalid section: .
        ...   Evaluating as NimScript file failed with: 
        ...     /tmp/nimblecache/nimscriptapi.nim(7, 8) Error: cannot open file: strformat
        ... printPkgInfo() failed.

I searched the .choosenim/toolchains/nim-1.2.0/lib/pure/strformat.nim was in place and was valid nim code. Then I looked at .choosenim/toolchains/nim-1.2.0/config/nim.cfg and was a plain binary data, also everything under dist/nimble/ and config/ is binary data instead of original nim code.

So I typed find . -type f -exec file '{}' \; | grep ": data" in .choosenim/toolchains/nim-1.2.0/ output for potential corrupted files and there were a lot so. I did the same on nim-1.0.6 toolchain and no output.

I don't know very well find behavior but nim.cfg should be ASCII Text.

Here is the most valuable output:

find . -type f -exec file '{}' \; | grep ": data"
./koch.nim: data
./config/nimdoc.cfg: data
./config/nimdoc.tex.cfg: data
./config/nim.cfg: data
...
./tests/... : data
...
./deinstall.sh: data
./dist/nimble ... : data
./install.sh: data

I fixed it temporarly copying the corrupted files from nim-1.2.0-linux_x64.tar.xz i downloaded manually. I hope this doesn't happen too often.

genotrance commented 4 years ago

What version of Arch are you running?

genotrance commented 4 years ago

Cc @voltist who report this in irc. Can you also please share your distro?

Also, does the issue only happen when installing 1.2.0 or can you repro it while installing 1.0.6 as well?

ghost commented 4 years ago

I'm also using Arch. FYI, Arch is rolling release, so 'version' doesn't have any useful meaning for troubleshooting like this. I updated all packages to the latest version literally moments before using choosenim, so the problem cannot be because of anything being outdated (perhaps some packages on my system are too new?).

It seems the issue is not present when Nim version 0.16.0 is installed. I remember now that when I first downloaded the choosenim install script (from the Nim website) and ran it, it didn't build anything. It just downloaded the 1.2.0 compiler and then said it was done.

genotrance commented 4 years ago

I just tried this with the latest Arch Linux docker image and could not reproduce the issue.

> docker run -it --rm archlinux:latest bash
> ... update, setup gcc
> curl https://nim-lang.org/choosenim/init.sh -sSf | sh

It installed Nim v1.2.0. I then ran find . -type f -exec file '{}' \; | grep ": data" but there was no output.

ghost commented 4 years ago

I am unable to recreate the issue on Manjaro Linux, so it must be some minor difference between the distros, or an issue with the impacted installs.

ghost commented 4 years ago

@genotrance I was using: Arch Linux x86_64 release 2020.04.01 now I updated the whole system. to 2020.05.01 Kernel: 5.6.10-arch1-1 or minor version (I don't remember the kernel version) I was wondering if there are some other reports about this. It keeps happening. However to be able to work i did stick with choosenim 0.4.0.

ghost commented 4 years ago

pacman -Syu nim nimble is perfectly useable as well, as long as you want to be running nothing but the latest stable version.

ghost commented 4 years ago

I tried the same procedure on the same desktop mentioned above using choosenim 0.5.1 and 0.6.0 for different toolchains, it corrupts them as well, I tried choosenim 0.6.0 and 0.5.1 on another computer and they work as expected, I have the same version of Arch Linux and same kernel (operating system is identical) on both. But the question is why choosenim 0.4.0 works fine on the desktop and laptop.

I would like to know if someone else has the same issue (corrupted toolchain).

I like choosenim and I am not planning to change my installation method.

genotrance commented 4 years ago

This is most likely an extraction issue - v0.5.0 of choosenim started using libarchive to extract all files and it is possible that the statically linked library is misbehaving on your system somehow. Either that or it is possible the archive is corrupted during download but that seems unlikely if it fails every time.

Would you be able to build the latest choosenim from scratch on your failing computer and see if that build works as expected? You will need to setup Nim v1.0.6, git clone, nimble install -d (to install deps) and then nimble build.

ghost commented 4 years ago

@genotrance I have cloned repositories with tags v0.4.0, v0.5.0, v0.5.1, and HEAD, I downloaded the dependencies with nimble install -d which I already had because I tried to install choosenim with nimble earlier. Then I build each one of them with nimble build. None of them built. Here is the most meaningful output for v0.5.0 and v0.5.1 and HEAD:

... # Resetting /home/ratriot/.cache/nim/nimterop/nimarchive/bzip2
... # Including library /home/ratriot/.cache/nim/nimterop/nimarchive/bzip2/libbz2.a
... Hint: lzma [Processing]
... # Including library /home/ratriot/.cache/nim/nimterop/nimarchive/liblzma/src/liblzma/.libs/liblzma.a
... Hint: zlib [Processing]
... # Including library /home/ratriot/.cache/nim/nimterop/nimarchive/zlib/libz.a
... # Running make -j 2
... #   Path: /home/ratriot/.cache/nim/nimterop/nimarchive/libarchive
... stack trace: (most recent call last)
... /home/ratriot/.nimble/pkgs/nimterop-0.4.4/nimterop/build.nim(1032, 23)
... /home/ratriot/.nimble/pkgs/nimterop-0.4.4/nimterop/build.nim(802, 9) buildLibrary
... /home/ratriot/.nimble/pkgs/nimterop-0.4.4/nimterop/build.nim(619, 18) make
... /home/ratriot/.nimble/pkgs/nimterop-0.4.4/nimterop/build.nim(105, 11) execAction
... /home/ratriot/.choosenim/toolchains/nim-1.0.6/lib/system/assertions.nim(27, 20) failedAssertImpl
... /home/ratriot/.choosenim/toolchains/nim-1.0.6/lib/system/assertions.nim(20, 11) raiseAssert
... /home/ratriot/.choosenim/toolchains/nim-1.0.6/lib/system/fatal.nim(39, 5) sysFatal
... /home/ratriot/.nimble/pkgs/nimarchive-0.3.6/nimarchive/archive.nim(83, 10) template/generic instantiation of `getHeader` from here
... /home/ratriot/.choosenim/toolchains/nim-1.0.6/lib/system/fatal.nim(39, 5) Error: unhandled exception: /home/ratriot/.nimble/pkgs/nimterop-0.4.4/nimterop/build.nim(105, 16) `false` Command failed: 2
... cmd: cd /home/ratriot/.cache/nim/nimterop/nimarchive/libarchive && make -j 2
... result:
... make  all-am
... make[1]: se entra en el directorio '/home/ratriot/.cache/nim/nimterop/nimarchive/libarchive'
...   CCLD     libarchive.la
... *** Warning: Linking the shared library libarchive.la against the
... *** static library /home/ratriot/.cache/nim/nimterop/nimarchive/liblzma/src/liblzma/.libs/liblzma.a is not portable!
... *** Warning: Linking the shared library libarchive.la against the
... *** static library /home/ratriot/.cache/nim/nimterop/nimarchive/zlib/libz.a is not portable!
... *** Warning: Linking the shared library libarchive.la against the
... *** static library /home/ratriot/.cache/nim/nimterop/nimarchive/bzip2/libbz2.a is not portable!
... /usr/bin/ld: /home/ratriot/.cache/nim/nimterop/nimarchive/zlib/libz.a(deflate.o): relocation R_X86_64_PC32 against symbol `z_errmsg' can not be used when making a shared object; recompile with -fPIC
... /usr/bin/ld: final link failed: bad value
... collect2: error: ld returned 1 exit status
... make[1]: *** [Makefile:4259: libarchive.la] Error 1
... make[1]: se sale del directorio '/home/ratriot/.cache/nim/nimterop/nimarchive/libarchive'
... make: *** [Makefile:3773: all] Error 2 [AssertionError]

This seems to be related with the statically linked library as you said. May be the way the library is compiled or how the linker is called. I hope this output gives you a clue: /usr/bin/ld: /home/ratriot/.cache/nim/nimterop/nimarchive/zlib/libz.a(deflate.o): relocation R_X86_64_PC32 against symbol `z_errmsg' can not be used when making a shared object; recompile with -fPIC

The choosenim v0.4.0 also failed but with choosenim_v0.4.0/src/choosenim/common.nim(4, 31) Error: undeclared identifier: 'NimbleError' but this is out of this issue context.

If you know how to fix this I would appreciate it. So we can close this issue.

dom96 / choosenim