search

riga / law

Build large-scale task workflows: luigi + job submission + remote targets + environment sandboxing using Docker/Singularity
http://law.readthedocs.io
BSD 3-Clause "New" or "Revised" License
98 stars 41 forks source link

Add option -limit to condor_history #86

Closed mschnepf closed 4 years ago

mschnepf commented 4 years ago

condor_history scans per default the complete history. The option -limit stops condor_history after it founds the given number. This reduces the time to check the status of old jobs.

riga commented 4 years ago

Hi @mschnepf , thanks for the PR, appreciate it! This is actually very useful as status querying can get really slow on some systems. I hope you don't mind that I added another commit to use the -limit option only for htcondor > v8.3.3. If you're fine with that change, I can merge.

riga commented 4 years ago

I just saw that condor_q has the same option, also since v8.3.3. Since we know the maximum number of jobs to expect in the response of condor_q (== len(job_ids)), I don't see why we shouldn't add -limit here as well. Objections?

mschnepf commented 4 years ago

Good idea to check which version of HTCondor supports this feature. It worked always for me. :-) I will add the -limit option also to HTCondor condor_q command.

riga commented 4 years ago

@mschnepf Looks like you committed a different local change.