search

nim-lang / Nim

Nim is a statically typed compiled systems programming language. It combines successful concepts from mature languages like Python, Ada and Modula. Its design focuses on efficiency, expressiveness, and elegance (in that order of priority).
https://nim-lang.org
Other
16.41k stars 1.47k forks source link

prefix vs. destination distinction for install.sh/koch install #9788

Closed dom96 closed 2 years ago

dom96 commented 5 years ago

The main pain point of install.sh and koch install is that it conflates the destination with the prefix. For most packagers it would be far more useful to specify a "prefix" alongside this...

>> PREFIX=/usr/bin DESTDIR=/tmp/my-package-build/ ./install.sh 
Creating /tmp/my-package-build/usr/bin/nim
....
FedericoCeratto commented 5 years ago

Related to #9784 and the need for documentation.

Araq commented 5 years ago

I don't understand it.

dom96 commented 5 years ago

@Araq If I run ./install.sh /tmp/my-package-build it'll create (something like this, I can't be bothered looking up exactly):

/tmp/my-package-build:
    bin/nim
    lib/nim

Where as I want:

/tmp/my-package-build:
  usr/bin/nim
  etc/nim/nim.cfg
  usr/lib/nim/stdlib.nimble

Does that make sense?

Araq commented 5 years ago

Ok, but if the prefix is usr how do I know this means etc/nim/nim.cfg and not usr/nim/nim.cfg or usr/etc/nim/nim-cfg? What kind of "prefix" is this madness?

zerothi commented 3 years ago

Ok, but if the prefix is usr how do I know this means etc/nim/nim.cfg and not usr/nim/nim.cfg or usr/etc/nim/nim-cfg? What kind of "prefix" is this madness?

I was too puzzled by this some time ago. Let me refer to this page

Basically DESTDIR is nice for staged builds. So I would really encourage you to enable this.

Also, it is quite confusing that

./koch install DIR

creates DIR/nim/ folder. When doing HPC installations using modules it would be nice if DIR is obeyed with no nim subdirectory. It isn't necessary and just conflates the purpose of bin, lib etc.

Araq commented 3 years ago

You're not supposed to use ./koch install at all, you are supposed to use your Linux package manager.

zerothi commented 3 years ago

You're not supposed to use ./koch install at all, you are supposed to use your Linux package manager.

But that won't work on a HPC system where you want to expose nim for end-users, and allowing them to have different versions installed.

Say scientist A uses nim 1.4.1 and has for quite some time for a research project, then 1.4.2 comes out and A tries it. But results are slightly different, for one reason or the other. So A decides to shift back to 1.4.1 for consistency/reproduceability.

Changing back is tedious for system-administrators.

Then one could argue that users should use choosenim for having a local installation, which I also think is sub-optimal. In the ideal world one can install nim in any location for explicit control to end-users.

Python has this supported to some extend, Julia is worse, but I think this would be nice for nim to have a good control.

This opens up problems related to user-installed packages using nimble. Say I use 1.4.1 and use nimble to install package B, then I switch to 1.4.2 and try to install B again. It would be ideal if they live in two different places so one can always switch between them; however, importantly these installed packages should live in $HOME, and not in the system-wide installation.

timotheecour commented 3 years ago

Then one could argue that users should use choosenim for having a local installation, which I also think is sub-optimal. In the ideal world one can install nim in any location for explicit control to end-users.

sh build_all.sh to build locally, choosenim 1.4.2 or choosenim path/to/nim to switch what the nim shim binary points to

This opens up problems related to user-installed packages using nimble

NIMBLE_DIR=$HOME/.nimble_test nimble develop nimquery

Maybe CHOOSENIM_DIR would be nice (like NIMBLE_DIR, to avoid global effects).

What else are you missing?

Araq commented 3 years ago

Say scientist A uses nim 1.4.1 and has for quite some time for a research project, then 1.4.2 comes out and A tries it. But results are slightly different, for one reason or the other. So A decides to shift back to 1.4.1 for consistency/reproduceability.

Changing back is tedious for system-administrators.

The scientiests don't have to contact the sys-admin at all. They can use the Nim from their home directory. It's what I do too.

zerothi commented 3 years ago

Then one could argue that users should use choosenim for having a local installation, which I also think is sub-optimal. In the ideal world one can install nim in any location for explicit control to end-users.

sh build_all.sh to build locally, choosenim 1.4.2 or choosenim path/to/nim to switch what the nim shim binary points to

We don't want to encourage all our scientists using nim to have the same executable installed in each of their HOME.

This opens up problems related to user-installed packages using nimble

NIMBLE_DIR=$HOME/.nimble_test nimble develop nimquery

My request would extend to having NIMBLE_DIR automatically having the version embedded. But that is a minor point that could easily be remedied through modules.

For instance python install stuff in .local/lib/python<MAJOR.MINOR>/ (excluding MICRO).

Maybe CHOOSENIM_DIR would be nice (like NIMBLE_DIR, to avoid global effects).

What else are you missing?

I am probably just missing the system-admin's installation process in a strict way. See here where there is a guide, but not for installing it in other places.

Changing back is tedious for system-administrators.

The scientiests don't have to contact the sys-admin at all. They can use the Nim from their home directory. It's what I do too.

This is not optimal for heterogeneous installations, then it requires users to install using the lowest denominator and loose performance for better hardware (SSE vs. avx vs avx2 vs avx512).
I really think this is a bad idea to put everything in the hands of scientists ;) We want to help end-users giving them the best performance, this can be controlled through modules and sys-wide base-packages.

Right now I get by this, quite easily by doing something like this:

sh build.sh
bin/nim c koch
./koch docs -d:release
./koch boot -d:release
./koch tools -d:release
./koch geninstall PREFIX
# this is needed since the above command always suffixes _any_ path with /nim
sed -i -e '/case/,/esac/{s:/nim::g}' install.sh
sh install.sh PREFIX

However, it seems hacking to fix the /nim suffix appends manually... I really don't like that ;)

Araq commented 3 years ago

Feel free to create a PR.

This is not optimal for heterogeneous installations, then it requires users to install using the lowest denominator and loose performance for better hardware (SSE vs. avx vs avx2 vs avx512).

I don't understand this, the Nim compiler delegates these things to the clang/gcc anyway.

But as I said, PRs are welcome.

zerothi commented 3 years ago

Feel free to create a PR.

I would be more than happy to. But I can't figure out where the /nim gets appended? I searched in the config files, to no avail...

Any hints would be appreciated. :+1:

This is not optimal for heterogeneous installations, then it requires users to install using the lowest denominator and loose performance for better hardware (SSE vs. avx vs avx2 vs avx512).

I don't understand this, the Nim compiler delegates these things to the clang/gcc anyway.

Isn't the stdlib pre-compiled and uses as is, this has been my understanding.. But that seems to be incorrect. I.e. is everything, all down to stdlib compiled on every nim invocation? That would make life much easier ;) Basically, are there any binaries/libraries that are shipped with nim that aren't used by other than the compiler?

Araq commented 3 years ago

IIRC it's encoded in tools/niminst/install.nimf the pattern /?proj (niminst can be used for different projects).

But that seems to be incorrect. I.e. is everything, all down to stdlib compiled on every nim invocation? That would make life much easier ;)

Yeah, everything is recompiled.