search

CoffeaTeam / lpcjobqueue

A dask-jobqueue plugin for the LPC Condor queue
BSD 3-Clause "New" or "Revised" License
12 stars 11 forks source link

Fallback to /etc/condor/condor_config #11

Open mapsacosta opened 2 years ago

mapsacosta commented 2 years ago

The Schedd class is only capable of using a condor_config file present in the same folder. In order to make this code friendlier with people running their own variations of condor, the line: https://github.com/CoffeaTeam/lpcjobqueue/blob/b08e3c7ae3ca24bf12888bcfcb5dbeaa5d697679/src/lpcjobqueue/schedd.py#L7 Should fallback to the default directory /etc/condor/condor_config and then fail.

The location of the condor_config could also be a configurable parameter

nsmith- commented 2 years ago

I think then we would want a separate docker image on top of docker-coffea-dask that adds this file, for use at LPC, right?

mapsacosta commented 2 years ago

Yep, in our case we just pull the file (and other condor configs) from Git with an initContainer. I could also see this as either a configurable parameter or an environment variable, especially if we're trying to extend to other places (such as lxplus). I am working on a quick MR to implement falling back to the default condor installation file and we can re-iterate if needed.

nsmith- commented 2 years ago

Sounds good. Then, what to do about the schedd picking algorithm? It currently is somewhat LPC-specific, idk what it would do at other sites.

nsmith- commented 1 year ago

As of #21 we now just always load the path https://github.com/CoffeaTeam/lpcjobqueue/blob/671334903c484f4e49e2c496efa1241bb4efd18a/src/lpcjobqueue/schedd.py#L8 Does that work for you, or should we do a if os.path.exists and then fallback to /etc/condor/condor_config? (actually if we just don't set the env variable, that is the default right?