Clarify location / role of chplconfig files when doing `./configure` && `make install`

bradcray commented 4 years ago

[Filing on behalf of @milisarge who went through a lot of confusion, with me in tow, over this]

As a Chapel programmer, it seems that the current documentation about chplconfig files, particularly with respect to their usage with ./configure and make install is incomplete, incorrect, and/or not as clear as it could be. Specifically:

https://chapel-lang.org/docs/usingchapel/building.html#installing-chapel indicates that ./configure will create a chplconfig file, but doesn't make it particularly clear what will happen to this file when make install is run (where the answer seems to be that it's copied somewhere that the installation then finds it? but where is this? where is the compiler getting it from and/or where should the user look for it?)
"The search priority..." text at https://chapel-lang.org/docs/usingchapel/chplenv.html#search-paths-and-file-names doesn't seem to list the above location as one of the potential options, only ~/ and $CHPL_HOME. Yet installed versions of Chapel seem to find the chplconfig that was created by ./configure + make install suggesting there must be another place that it's searched for that should be added to this list in the appropriate spot?
Related: it's not entirely clear what $CHPL_HOME is for a Chapel installation built with ./configure + make install
The "Variable priority" text at https://chapel-lang.org/docs/usingchapel/chplenv.html#search-paths-and-file-names only lists ~/.chplconfig and doesn't suggest other potential locations of chplconfig files. It seems like this should be less specific and/or refer to the "search priority" text above. The current text suggests that ~/.chplconfig is the answer, not just a sample of where the value might come from.

bradcray commented 4 years ago

@ben-albrecht and @mppf: Would one of you take ownership of this, as I think you're most familiar with these features and this documentation?

ben-albrecht commented 4 years ago

I can help with this issue.

where the answer seems to be that it's copied somewhere that the installation then finds it?

If I understand correctly, it creates the chplconfig in the same directory as configure ($CHPL_HOME). Agree that this documentation could be clearer.

The search priority doesn't seem to list the above location as one of the potential options,

I think this text is still correct (source). It must be finding the chplconfig generated in $CHPL_HOME for the make install case.

It seems like this should be less specific and/or refer to the "search priority" text above. The current text suggests that ~/.chplconfig is the answer, not just a sample of where the value might come from.

Agreed.

bradcray commented 4 years ago

If I understand correctly, it creates the chplconfig in the same directory as configure ($CHPL_HOME).

I agree with that part, but the question is what does make install do with the chplconfig file that ./configure creates. Note that the original $CHPL_HOME is likely to be deleted and should no longer be required after make install. Our impression is that make install copies the chplconfig somewhere else.

It must be finding the chplconfig generated in $CHPL_HOME for the make install case.

I don't believe that's the case once make install has been run (edit: or, at least, not the original $CHPL_HOME location from which the installation was built. This relates to my question in the third bullet regarding what CHPL_HOME is considered to be for a make install-generated installation of Chapel. It doesn't seem to need to be set by the user, yet it's not entirely clear to me what it's considered to be for the purposes of things like finding chplconfig files).

ben-albrecht commented 4 years ago

Ah, found it in compiler/main/driver.cpp:

https://github.com/chapel-lang/chapel/blob/80832f088a652ff79de74e10a37c987073234d63/compiler/main/driver.cpp#L460-L480

I think not establishing precedence for this check vs. the other locations was an oversight, and the lack of documentation follows that.

mppf commented 4 years ago

Our impression is that make install copies the chplconfig somewhere else.

It does:

https://github.com/chapel-lang/chapel/blob/80832f088a652ff79de74e10a37c987073234d63/util/buildRelease/install.sh#L349-L358

For the install-to-a-directory option, I'd imagine that that directory becomes CHPL_HOME and as a result the chplconfig is used... but for the --prefix install option there might be some other logic needed.

What exactly happens here in driver.cpp is not currently clear to me but I'd appreciate it if @ben-albrecht can handle it

milisarge commented 4 years ago

i want to give some details about my last build, (https://github.com/chapel-lang/chapel/commit/048da34ae3c98d2a1905489b5f1381f1118fdc60). i built with

export CHPL_TASKS=fifo
./configure --prefix=/usr
make
make install

and i have two config files, first one is /usr/lib/chapel/1.21/runtime/lib/linux64/gnu/x86_64/arch-native/loc-flat/comm-none/tasks-fifo/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/none/re2/fs-none/lib_pic-none/chplconfig its content:

CHPL_TARGET_PLATFORM=linux64
CHPL_TARGET_COMPILER=gnu
CHPL_TARGET_ARCH=x86_64
CHPL_TARGET_CPU=native
CHPL_LOCALE_MODEL=flat
CHPL_COMM=none
CHPL_COMM_SUBSTRATE=none
CHPL_GASNET_SEGMENT=none
CHPL_TASKS=fifo
CHPL_TIMERS=generic
CHPL_UNWIND=none
CHPL_MEM=jemalloc
CHPL_ATOMICS=cstdlib
CHPL_HWLOC=none
CHPL_REGEXP=re2
CHPL_AUX_FILESYS=none
CHPL_LIB_PIC=none

second one is /usr/lib/chapel/1.21/chplconfig, its content:

CHPL_TASKS=fifo

but chplprintenv output is diffrent: # /usr/share/chapel/1.21/util/printchplenv

machine info: Linux milistgn 5.4.14-milis #1 ZEN SMP PREEMPT Sat Jan 25 22:24:13 CET 2020 x86_64
CHPL_HOME: /usr/share/chapel/1.21
script location: /usr/share/chapel/1.21/util/chplenv
CHPL_TARGET_PLATFORM: linux64
CHPL_TARGET_COMPILER: gnu
CHPL_TARGET_ARCH: x86_64
CHPL_TARGET_CPU: native
CHPL_LOCALE_MODEL: flat
CHPL_COMM: none
CHPL_TASKS: qthreads
CHPL_LAUNCHER: none
CHPL_TIMERS: generic
CHPL_UNWIND: none
CHPL_MEM: jemalloc
CHPL_ATOMICS: cstdlib
CHPL_GMP: none
CHPL_HWLOC: hwloc
CHPL_REGEXP: none
CHPL_LLVM: none
CHPL_AUX_FILESYS: none

-- # /usr/share/chapel/1.21/util/printchplenv --internal

machine info: Linux milistgn 5.4.14-milis #1 ZEN SMP PREEMPT Sat Jan 25 22:24:13 CET 2020 x86_64
CHPL_HOME: /usr/share/chapel/1.21
script location: /usr/share/chapel/1.21/util/chplenv
CHPL_HOST_CPU: none
CHPL_ORIG_TARGET_COMPILER: gnu
CHPL_RUNTIME_CPU: native
CHPL_TARGET_CPU_FLAG: arch
CHPL_TARGET_BACKEND_CPU: native
  CHPL_LIBFABRIC: none
CHPL_HOST_MEM: cstdlib
CHPL_TARGET_MEM: jemalloc
  CHPL_JEMALLOC: jemalloc
CHPL_MAKE: make
CHPL_GMP: none
CHPL_RUNTIME_SUBDIR: linux64/gnu/x86_64/arch-native/loc-flat/comm-none/tasks-qthreads/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/hwloc/none/fs-none/lib_pic-none
CHPL_LAUNCHER_SUBDIR: linux64/gnu/x86_64/loc-flat/comm-none/tasks-qthreads/launch-none/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/lib_pic-none
CHPL_COMPILER_SUBDIR: linux64/gnu/x86_64/llvm-none
CHPL_HOST_BIN_SUBDIR: linux64-x86_64
CHPL_TARGET_BIN_SUBDIR: linux64-x86_64-native
  CHPL_LLVM_UNIQ_CFG_PATH: none
  CHPL_GASNET_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none/substrate-none/seg-none
  CHPL_GMP_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_HWLOC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none-flat
  CHPL_JEMALLOC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_LIBFABRIC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_LIBUNWIND_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_QTHREAD_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none-flat-jemalloc-hwloc
  CHPL_RE2_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_THIRD_PARTY_COMPILE_ARGS: 
  CHPL_THIRD_PARTY_LINK_ARGS: -lhwloc -ljemalloc -lqthread

about output of hello app: (writeln("hello from milis linux")) # ./hello --about

Compilation command: chpl hello.chpl 
Chapel compiler version: 1.21.0 pre-release (048da34ae3)
Chapel environment:
  CHPL_HOME: /usr/share/chapel/1.21
  CHPL_ATOMICS: cstdlib
  CHPL_AUX_FILESYS: none
  CHPL_COMM: none
  CHPL_COMM_SUBSTRATE: none
  CHPL_COMPILER_SUBDIR: linux64/gnu/x86_64/llvm-none
  CHPL_GASNET_SEGMENT: none
  CHPL_GASNET_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none/substrate-none/seg-none
  CHPL_GMP: gmp
  CHPL_GMP_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_HOST_ARCH: x86_64
  CHPL_HOST_BIN_SUBDIR: linux64-x86_64
  CHPL_HOST_COMPILER: gnu
  CHPL_HOST_CPU: none
  CHPL_HOST_MEM: cstdlib
  CHPL_HOST_PLATFORM: linux64
  CHPL_HWLOC: none
  CHPL_HWLOC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none-flat
  CHPL_JEMALLOC: jemalloc
  CHPL_JEMALLOC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_LAUNCHER: none
  CHPL_LAUNCHER_SUBDIR: linux64/gnu/x86_64/loc-flat/comm-none/tasks-fifo/launch-none/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/lib_pic-none
  CHPL_LIBFABRIC: none
  CHPL_LIBFABRIC_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_LIBUNWIND_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_LIB_PIC: none
  CHPL_LLVM: none
  CHPL_LLVM_UNIQ_CFG_PATH: none
  CHPL_LOCALE_MODEL: flat
  CHPL_MAKE: make
  CHPL_MEM: jemalloc
  CHPL_NETWORK_ATOMICS: none
  CHPL_ORIG_TARGET_COMPILER: gnu
  CHPL_QTHREAD_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none-flat-jemalloc-none
  CHPL_RE2_UNIQ_CFG_PATH: linux64-x86_64-native-gnu-none
  CHPL_REGEXP: re2
  CHPL_RUNTIME_CPU: native
  CHPL_RUNTIME_INCL: /usr//lib/chapel/1.21/runtime/include
  CHPL_RUNTIME_LIB: /usr//lib/chapel/1.21/runtime/lib
  CHPL_RUNTIME_SUBDIR: linux64/gnu/x86_64/arch-native/loc-flat/comm-none/tasks-fifo/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/none/re2/fs-none/lib_pic-none
  CHPL_TARGET_ARCH: x86_64
  CHPL_TARGET_BACKEND_CPU: native
  CHPL_TARGET_BIN_SUBDIR: linux64-x86_64-native
  CHPL_TARGET_COMPILER: gnu
  CHPL_TARGET_CPU: native
  CHPL_TARGET_CPU_FLAG: arch
  CHPL_TARGET_MEM: jemalloc
  CHPL_TARGET_PLATFORM: linux64
  CHPL_TASKS: fifo
  CHPL_THIRD_PARTY: /usr//lib/chapel/1.21/third-party
  CHPL_THIRD_PARTY_COMPILE_ARGS: 
  CHPL_THIRD_PARTY_LINK_ARGS: -lgmp -ljemalloc -lm -lre2 -lpthread
  CHPL_TIMERS: generic
  CHPL_UNWIND: none

milisarge commented 4 years ago

additionally, when i want to make new building by extracting source code. It still looks to my local installed chapel configuration. For example at me tasks is fifo, then new source also is built with fifo. it should be qthreads because of qthreads come defaultly as task

jabraham17 commented 1 month ago

resolved by https://github.com/chapel-lang/chapel/pull/25987

chapel-lang / chapel

Clarify location / role of chplconfig files when doing `./configure` && `make install` #14932