search

dmwm / CRABServer

15 stars 38 forks source link

unify getPfn functions and Rucio client use in TW #6278

Open belforte opened 3 years ago

belforte commented 3 years ago

there are now two almost indentical utility functions to resolve LFN to PFN used in TW

https://github.com/dmwm/CRABServer/blob/32066a9248142e7851ebf9ebe0dd12f95679bef4/src/python/TaskWorker/Actions/StageoutCheck.py#L28 and https://github.com/dmwm/CRABServer/blob/32066a9248142e7851ebf9ebe0dd12f95679bef4/src/python/TaskWorker/Actions/DagmanCreator.py#L371

should rather have a single one e/g/ in ServerUtilities and actually the Rucio client initialization should be done once per slave instance and reused, in particular in DBSDataDiscovery.

belforte commented 3 years ago

maybe best is to create a Rucio client early in the Handler and pass it around like done currently for the REST server, it is not optimal that code is replicated in HandleNewTask, HandleResubmit and HandleKill Also I wonder if the Handler is anyhow closed at the end of the task processing and a new server object instantiated for any new task... maybe REST server and Rucio client should be part of the Worker hmm maybe best is to add a RESTServer and RucioClient objects to the config, which is already passed around everywhere minimizing the risks of messing up things