Snakemake-Profiles / lsf

Snakemake profile for running jobs on an LSF cluster
MIT License
36 stars 22 forks source link

Cluster cancel doesn't work with current job ID #55

Closed mbhall88 closed 2 years ago

mbhall88 commented 2 years ago

Because we emit the log path and the job ID together, setting cluster cancel to bkill fails with

Terminating processes on user request, this might take some time.
Traceback (most recent call last):
  File "/homes/mbhall88/.config/snakemake/lsf/lsf_status.py", line 232, in <module>
    print(lsf_status_checker.get_status())
  File "/homes/mbhall88/.config/snakemake/lsf/lsf_status.py", line 156, in get_status
    status = self._query_status_using_bjobs()
  File "/homes/mbhall88/.config/snakemake/lsf/lsf_status.py", line 99, in _query_status_using_bjobs
    output_stream, error_stream = OSLayer.run_process(self.bjobs_query_cmd)
  File "/homes/mbhall88/.config/snakemake/lsf/OSLayer.py", line 40, in run_process
    completed_process = subprocess.run(
  File "/hps/software/users/iqbal/mbhall/miniconda3/envs/who-correspondence/lib/python3.10/subprocess.py", line 503, in run
    stdout, stderr = process.communicate(input, timeout=timeout)
  File "/hps/software/users/iqbal/mbhall/miniconda3/envs/who-correspondence/lib/python3.10/subprocess.py", line 1149, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/hps/software/users/iqbal/mbhall/miniconda3/envs/who-correspondence/lib/python3.10/subprocess.py", line 2000, in _communicate
    ready = selector.select(timeout)
  File "/hps/software/users/iqbal/mbhall/miniconda3/envs/who-correspondence/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
3399781 logs/cluster/download_data/proj=PRJNA650381.sample=SAMN27765622.run=SRR18914164/jobid258_ff07f69d-f434-4b59-99f1-3f65c066bf08.out: Illegal job ID.
1 out of 1 calls to --cluster-cancel failed.  This is safe to ignore in most cases.

I guess we just can't emit the log path with the job ID.

@leoisl maybe we could emit the log path to stderr?