search

WIPACrepo / pyglidein

Some python scripts to launch HTCondor glideins
MIT License
7 stars 20 forks source link

Make partitions for SLURM configurable. Resolve #29. #30

Closed briedel closed 8 years ago

dsschult commented 8 years ago

I'm not sure the cleanup code is generalized enough - it seems specific to one batch system.

But I will take the last two commits as a valid solution to #29.

briedel commented 8 years ago

Generalizing the cleanup code is hard. It heavily depends on how the batch system reports the jobs ids and how the temp directory is setup. Do you have a good solution?

dsschult commented 8 years ago

Maybe move the cleanup code into the submit classes?

briedel commented 8 years ago

How is submitting relate to cleanup after a job is complete? One could keep a list of jobs id that were submitted and the time they were submitted. Once they hit the walltime the folder will be deleted

dsschult commented 8 years ago

They're related in that they both interact with the local batch system. vs client.py is abstracted away from that.

briedel commented 8 years ago

Okay, adding it to submit() after all the jobs have been submitted might be the better solution in that case.

dsschult commented 8 years ago

I don't mean that it has to be part of submit(), but maybe another function in the classes inside submit.py.

briedel commented 8 years ago

I realize. It should happen either before or after submit() submits the jobs. The best option may be to make it a function inside the Submitxxx class and let it be called from client.py

dsschult commented 8 years ago

missing a default cleanup (no-op) on the main Submit class. otherwise looks good.

briedel commented 8 years ago

Added