Sometimes "opam install" wants to recompile many packages - after Ctrl+C + retry it won't

[MSoegtropIMC]

I have an odd issue: I have scripts which call opam install with a largish number of packages (~50) to leverage parallel install. Sometimes opam then wants to recompile a good fraction of the packages, even though they are already installed and to my best knowledge nothing changed (I do run opam update before). The very odd thing is that when I abort the opam install with Ctrl+C and try the same command again, it doesn't try to reinstall and just says that all packages are already installed. I appended a log.

superflous_update.log

I have the impression that this started after I changed my scripts to switch to a certain switch with

eval $(opam env --set-switch --switch ${COQ_PLATFORM_SWITCH_NAME})

rather than

opam switch ${COQ_PLATFORM_SWITCH_NAME}

which I did because I sometimes run several opam installs on different switches in different shells. Should I maybe do both?

coq-platform-main$ opam config report
# opam config report
# opam-version         2.1.0 
# self-upgrade         no
# system               arch=x86_64 os=macos os-distribution=macports os-version=11.6.2
# solver               builtin-mccs+glpk
# install-criteria     -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed
# upgrade-criteria     -removed,-count[avoid-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new
# jobs                 15
# repositories         4 (http), 11 (local) (default repo at b3f8647a)
# pinned               0
# current-switch       __coq-platform.2022.01.0~8.15~preview1
# ocaml:native         true
# ocaml:native-tools   true
# ocaml:native-dynlink true
# ocaml:stubsdir       /Users/msoegtrop/.opam/__coq-platform.2022.01.0~8.15~preview1/lib/ocaml/stublibs:/Users/msoegtrop/.opam/__coq-platform.2022.01.0~8.15~preview1/lib/ocaml
# ocaml:preinstalled   false
# ocaml:compiler       4.10.2

[kit-ty-kate]

Your log doesn’t include the opam commands being executed. Could you send a link to this ./coq_platform_make.sh script?

[MSoegtropIMC]

The script is rather lengthy (~ 20 files). The call which is failing is here:

https://github.com/coq/platform/blob/18992d85bc2df3345fb0d71c2fcf5a4c9d15b272/shell_scripts/build.sh#L35 - a plan opam install <some packages.

The variable "${PACKAGES//PIN.}" is defined here: https://github.com/coq/platform/blob/995ccb5563e820f6fd2fa84f4c068a6eda2e2a08/package_picks/package-pick-8.15%7Epreview1.sh

Note that the PIN stuff is ignored in a parallel install.

[dra27]

Some code forensics:

• The "Package foo is already installed" message is correctly not displaying for packages being recompiled
• It's not clear why the packages have been marked for recompilation - because the packages are explicitly being installed, opam doesn't display the reason for recompilation (because it assumes you asked for them and know why)
• What is not yet clear to us is why the packages don't get recompiled on subsequent installations

[MSoegtropIMC]

because the packages are explicitly being installed, opam doesn't display the reason for recompilation

Is there a way to show the reasons anyway? Maybe with -v?