search

iris-hep / analysis-grand-challenge

Repository dedicated to AGC preparations & execution
https://agc.readthedocs.io
MIT License
24 stars 39 forks source link

Running the CMS workload from LCG_10X #160

Open sciaba opened 1 year ago

sciaba commented 1 year ago

To be able to run the workload from SWAN using Dask to scale out to HTCondor, ideally all dependencies should be satisfied by a given LCG configuration, as the remote Dask workers will have that. Would it make sense to create the minimal requirements.txt to be used on top of the latest LCG configuration (103) to enable this use case?

matthewfeickert commented 1 year ago

Ideally all dependencies should be satisfied by a given LCG configuration

I don't think this is a reasonable thing to try to do. LCG views are (in my view) for Python a bit of a nightmare as they become environments that are not well defined in terms of requirements and fragile to build upon.

While I'm sure this is possible to do, I think it would just be painful and lead to lots of headaches compared to just fully defining the environment.