Star dev Library: Include required spack libraries

[jdbrice]

Include required spack libraries (32bit and 64bit) into standard STAR library (i.e. DEV for now)

The StFwdTrackMaker and other Fwd makers require various libraries installed in spack (GenFit, Rave, KiTrack). So far we have been using a manually constructed environment with these overlaid on dev (64 bit).

See https://github.com/jdbrice/star-sw-1/blob/fwd-tracking/StRoot/StFwdTrackMaker/macro/env.sh

NOTE: we have a production request incoming that will require this to be in place to proceed

[plexoos]

This is something the Production team would need to address by incorporating into their scripts.

[jdbrice]

@plexoos I added Gene, if you think others should be added let me know (or feel free to add them)

[genevb]

This appears to me to be a STAR computing environment topic, which may need help from Jerome, not the Production Team.

-Gene

[jdbrice]

Hi all,

What can I do to move this forward then? I have meeting with the FWD upgrade people tomorrow. We also need to start the calibration production that this is needed for asap.

[plexoos]

Hey Daniel,

In case you don't hear back from those managing the cluster environment/builds in a timely manner, let me suggest another option based on the already available containers. It is quite straightforward to run a root4star job in a singularity container, in fact, for the latest code on the main branch it is just these two lines:

$ singularity build star-sw-root5.sif docker://ghcr.io/star-bnl/star-sw-root5-base:main_latest                                                                                                                                                              
$ singularity run -e -B /gpfs01:/gpfs01 star-sw-root5.sif root4star -b -q 'bfc.C(10, \"P2018a,StiCA,btof,mtd,PicoVtxDefault,BEmcChkStat,QAalltrigs,OSpaceZ2,OGridLeak3D,-hitfilt\", \"/gpfs01/star/rcf/test/daq/2018/136/st_physics_19136012_raw_1000016.daq\")'

[genevb]

What can I do to move this forward then?

I mentioned that Jerome might be able to help, if you want to try contacting him.

-Gene

[plexoos]

During the last meeting we discussed the possibility of adding a new test for the forward track maker. If nothing changed, I still think we should add it to the list. All we need is just a list of options for bfc.C and an input file.

[jdbrice]

Hi @plexoos I am happy to add this, however I am still waiting on others handle the main request here. I realized after talking - even if all of this runs correctly in the CI system (running in containers) it will not work on RCF and will be broken in DEV.

However, seeing the list helps me prepare this. Quick question: what is used to determine success / failure? is it just the exit status of the BFC?

[veprbl]

However, seeing the list helps me prepare this. Quick question: what is used to determine success / failure? is it just the exit status of the BFC?

I can answer this one. You can't trust the exit code of ROOT5, instead we look at the log output: https://github.com/star-bnl/star-sw/blob/87920ac14b2a866c678b25648caeba9f214cbb75/.github/workflows/build-pull-request.yml#L56 Ideally, we should check that the generated files are valid, but that's to be implemented yet.

[jdbrice]

Thanks @veprbl, I can use this to setup a few tests myself so that I am ready to add them once everything else is in place

[plexoos]

Great ...and the input file(s) for the new tests will be added to this image https://github.com/star-bnl/star-sw/pkgs/container/star-test-data (I can help with that). The only request is to keep the input files relatively small if possible, something on the order of a few MBs, so more tests can be accommodated into a single image if needed in the future.

[plexoos]

I believe this comment from Jerome belongs here:

A basic scheme is implemented but would need to be tested. Essentially, starver was extended as follows 1- starver SL22a would do as usual while starver SL22a bla would also source $STAR/mgr/bla.config - such a file may contain anything you wish. It would know about about the USE_64BITS env so one could load the 32bits or 64bits spack depending on that. This would allow for an infinite number of configs ... however

2- starver SL22a would ALSO source a file named $STAR/mgr/default.config This is to address the discussion that for a given library, there may actually not be multiple config.

The syntax also allows for $ starver SL22a-bla This is to facilitate the use of production wrapper scripts AND allow for a Cataloguing that may include that as a reference (the path where the files are produced should also reflect the fill library version). But this is a production team issue ...

So, you have the hooks to move forward. I am leaving / taking a break so will get get back to it if people had problems moving from this point onward.

Someone need to write one of those "config" files (a csh script really but gave a specific name to make it clear). For now, only $STAR/mgr/ location is supported but I can extend to the local user dir if needed.

One more comment, if some form of alias for loading dozen of packages is made in spack. it MUST be unique or again, we risk to get into a realm of not being able to reproduce past results (an environment needs to be uniquely defined).