search

star-bnl / star-sw

Core software for STAR experiment
26 stars 63 forks source link

Add scripts to load dependency packages via Environment Modules #409

Closed jdbrice closed 1 year ago

jdbrice commented 1 year ago

This PR provides 2 configs to load spack library versions: 0.2.3 (32b & 64b)

I have not added scripts for root 6 variants yet as I do not think these are used by anyone. If I understand correctly, these will become usable with, e.g.:

starver v0.2.3-rhel7-root5.34.38-32b

OR

starver v0.2.3-rhel7-root5.34.38-64b

Closes #359

klendathu2k commented 1 year ago

Did you mean to write starver dev v0.2.0-5.34.38?

Or is there a star script (sitting somewhere out of my search path) which takes the option ver?

plexoos commented 1 year ago

Did you mean to write starver dev v0.2.0-5.34.38?

Yes, it should be starver (no space) as explained in this comment

klendathu2k commented 1 year ago

Did you mean to write starver dev v0.2.0-5.34.38?

Yes, it should be starver (no space) as explained in this comment

Thanks. Also in that comment (and reading the code...) starver expects a filename like v0.2.0-5.34.38-64b.config not v0.2.0-5.34.38-64b.csh. Or am I missing something?

genevb commented 1 year ago

There would need to be a directory layout naming convention...

star-environment/root-version/ ... /32b.config or 64b.config

Other subdirectories to be added for (e.g.) geant versioning.

I appreciate the suggestion to minimize clutter, @klendathu2k .Can we know in advance what all items need versioning (e.g. GEANT, then what else?)....because re-organizing later would be problematic for maintaining compatibility of people's private scripts.

klendathu2k commented 1 year ago

I appreciate the suggestion to minimize clutter, @klendathu2k .Can we know in advance what all items need versioning (e.g. GEANT, then what else?)....because re-organizing later would be problematic for maintaining compatibility of people's private scripts.

@genevb Geant versioning would be good. But as we add more and more directory levels... the starver command becomes less and less user friendly. More error prone for users to discover the available scripts. More annoying to type. So perhaps a flat directory structure is preferred after all. But the scripts should be under mgr/config instead of mgr to avoid the clutter.

klendathu2k commented 1 year ago

And one last comment. The starver script looks for these environment scripts under $STAR/mgr. If it isn't found, a default config is executed. This means there is no possibility to specify a local script. The trick that I normally use (setting STAR to the local directory) doesn't work... b/c starver resets it.

$ ls mgr/mylocal.config
mgr/mylocal.config
$ setenv STAR .
$ starver dev mylocalversion
$ echo $STAR
/afs/rhic.bnl.gov/star/packages/DEV
jdbrice commented 1 year ago

And one last comment. The starver script looks for these environment scripts under $STAR/mgr. If it isn't found, a default config is executed. This means there is no possibility to specify a local script. The trick that I normally use (setting STAR to the local directory) doesn't work... b/c starver resets it.

$ ls mgr/mylocal.config
mgr/mylocal.config
$ setenv STAR .
$ starver dev mylocalversion
$ echo $STAR
/afs/rhic.bnl.gov/star/packages/DEV

Right, Jerome mentioned at the beginning that he would add this if we requested. Maybe we should request, as it would also help test this before final integration

jdbrice commented 1 year ago

Ok I sent an email to Jerome to address the loading of configs from local user directories. But we need to converge on the naming and backward compatibility issues as well. Commit b9a24df6e635d6791322b48165288eb4c2b17ea9 moves them into config sub directory and attempts to name them in a verbose way - but this is not user friendly at all

plexoos commented 1 year ago

I would drop the v0.2.0 configs and only keep the currently latest v0.2.3. Since it is going to be the first time we rely on this environment there is no need to keep the older ones around.

jdbrice commented 1 year ago

OK commit 35dd3b40875d94ffe4cc7af33c3123dfd18e0fee removes the v0.2.0 configs and removes geant3 from the name per @plexoos comments.

Also @plexoos - we talked about adding an alias as "default" or "current" etc. but does that need to be part of the PR?

Please let me know if if there are any other changes needed before approval

jdbrice commented 1 year ago

Hi can we resolve this - I have not received any new comments? We are waiting on this for the potential to run production with official libraries (or dev). Maybe just need an approval from @plexoos

I have not heard from Jerome, but will ping him.

plexoos commented 1 year ago

I have no objections and can approve and merge ASAP. If I understand correctly this will not have an immediately effect as there is no mgr/config/default.config but it is probably a good thing at this point. It would be nice to hear from the production team

genevb commented 1 year ago

I also have no objection with these files in a config subdirectory, and no impact on official productions.

It would be nice to hear from the production team

Waiting only 19 minutes to merge after requesting to hear from the Production Team seems a bit....short? ;-)

-Gene

p.s. Jerome is at a conference for which he is an organizer and contributing author on multiple talks. I'm not saying whether or not you will hear from him, but I'm giving you some context for why you might not.

plexoos commented 1 year ago

Waiting only 19 minutes to merge after requesting to hear from the Production Team seems a bit....short? ;-)

Depends on your reference point: 19 minutes since my comment or 13 days since the original post

Thank you for confirming the (lack of) possible impact!

klendathu2k commented 1 year ago

I checked out the group directory from CVS and modified .starver to look for a local configuration file. I confirmed that my version of starver runs, and executes the configuration script. I expect that the environment will be setup for root 6. It still points to root 5. So... this does not appear ready for prime time.

This is the config file which will be executed...

$ cat mgr/v0.23-6.16.00.config 
#!/bin/csh -f
echo Setting spack env
module use /cvmfs/star.sdcc.bnl.gov/star-spack/spack/share/spack/modules/linux-rhel7-x86_64/
module load star-env-0.2.3-root-6.16.00
module load zlib-1.2.12
module load root-6.16.00
module list

This is the starver command executed and its result (I've deleted most of the module list output because of formatting on the git page...)

starver DEV v0.23-6.16.00
Setting spack env
Currently Loaded Modulefiles:
  1) /boost-1.59.0   ...    14) /root-6.16.00    ....

And now which version of ROOT is loaded?

$ which root.exe
/afs/rhic.bnl.gov/star/ROOT/5.34.38/.sl73_x8664_gcc485/rootdeb/bin/root.exe
plexoos commented 1 year ago

I am a bit confused... What commands do I need to run to reproduce?

klendathu2k commented 1 year ago

Probably the simplest way to reproduce ...

$ cp -R ~jwebb/group . $ cp -R /star/simu/simu/jwebb/2022/star-root6/star-sw-1/mgr .

$ setup 64b $ setenv GROUP_DIR ./group $ starver dev v0.23-6.16.00

On 2022-10-26 10:58, Dmitri Smirnov wrote:

I am a bit confused... What commands do I need to run to reproduce?

-- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you were mentioned.Message ID: @.***>

Links:

[1] https://github.com/star-bnl/star-sw/pull/409#issuecomment-1292183077 [2] https://github.com/notifications/unsubscribe-auth/ANL4LVDUCDCMHKKI4CQ3CX3WFFBJLANCNFSM6AAAAAARDJOAX4

plexoos commented 1 year ago

Ok, I followed the steps exactly. Here is what I see:

$ which root.exe
/cvmfs/star.sdcc.bnl.gov/star-spack/spack/opt/spack/linux-rhel7-x86_64/gcc-4.8.5/root-6.16.00-bdclikaa3lfpfqolnfmwkcv5ws4vtepn/bin/root.exe
klendathu2k commented 1 year ago

Okay. That means there is something funny going on in my environment.

On 2022-10-26 11:36, Dmitri Smirnov wrote:

Ok, I followed the steps exactly. Here is what I see:

$ which root.exe /cvmfs/star.sdcc.bnl.gov/star-spack/spack/opt/spack/linux-rhel7-x86_64/gcc-4.8.5/root-6.16.00-bdclikaa3lfpfqolnfmwkcv5ws4vtepn/bin/root.exe

-- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you were mentioned.Message ID: @.***>

Links:

[1] https://github.com/star-bnl/star-sw/pull/409#issuecomment-1292236072 [2] https://github.com/notifications/unsubscribe-auth/ANL4LVB52C7WV3HRXTFS2LTWFFFYXANCNFSM6AAAAAARDJOAX4

klendathu2k commented 1 year ago

... and now it is working for me. I picked the wrong week to stop drinking coffee.

On 2022-10-26 11:47, jwebb wrote:

Okay. That means there is something funny going on in my environment.

On 2022-10-26 11:36, Dmitri Smirnov wrote:

Ok, I followed the steps exactly. Here is what I see:

$ which root.exe /cvmfs/star.sdcc.bnl.gov/star-spack/spack/opt/spack/linux-rhel7-x86_64/gcc-4.8.5/root-6.16.00-bdclikaa3lfpfqolnfmwkcv5ws4vtepn/bin/root.exe

-- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you were mentioned.Message ID: @.***>

Links:

[1] https://github.com/star-bnl/star-sw/pull/409#issuecomment-1292236072 [2] https://github.com/notifications/unsubscribe-auth/ANL4LVB52C7WV3HRXTFS2LTWFFFYXANCNFSM6AAAAAARDJOAX4