ComparativeGenomicsToolkit / cactus

Official home of genome aligner based upon notion of Cactus graphs
Other
528 stars 111 forks source link

TypeError issue in minigraph construct #1473

Closed NyagaM closed 2 months ago

NyagaM commented 2 months ago

Hi, Thanks for the pipeline, I have been using it to construct pangenomes of de novo assembled human assemblies. I am encountering TypeError issue TypeError: minigraph_construct() got an unexpected keyword argument 'has_resources' at process minigraph_construct when running Cactus 2.9.0 from the docker container. The is no issue when running Cactus 2.8.4. This is the error:

[2024-09-05T18:46:27+1200] [MainThread] [I] [toil.leader] Issued job 'minigraph_construct' kind-minigraph_construct/instance-wev60jsz v1 with job batch system ID: 13 and disk: 21.7 Gi, memory: 279.4 Gi, cores: 48, accelerators: [], preemptible: False
Exception ignored in: <function DeferredFunctionManager.__del__ at 0x7f97973d0ca0>
Traceback (most recent call last):
  File "/home/cactus/cactus_env/lib/python3.10/site-packages/toil/deferred.py", line 160, in __del__
    fcntl.lockf(self.stateFD, fcntl.LOCK_UN)
OSError: [Errno 5] Input/output error
[2024-09-05T18:46:28+1200] [Thread-1 (daddy)] [E] [toil.batchSystems.singleMachine] Got exit code 1 (indicating failure) from job _toil_worker minigraph_construct file:/denovo_assembled_genomes/trio/pangenome/js kind-minigraph_construc>
[2024-09-05T18:46:28+1200] [MainThread] [W] [toil.leader] Job failed with exit value 1: 'minigraph_construct' kind-minigraph_construct/instance-wev60jsz v1
Exit reason: None
[2024-09-05T18:46:28+1200] [MainThread] [W] [toil.leader] The job seems to have left a log file, indicating failure: 'minigraph_construct' kind-minigraph_construct/instance-wev60jsz v2
[2024-09-05T18:46:28+1200] [MainThread] [W] [toil.leader] Log from job "kind-minigraph_construct/instance-wev60jsz" follows:
=========>
        [2024-09-05T18:46:28+1200] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
        [2024-09-05T18:46:28+1200] [MainThread] [I] [toil] Running Toil version 7.0.0-d569ea5711eb310ffd5703803f7250ebf7c19576 on host liggen3dprd05.
        [2024-09-05T18:46:28+1200] [MainThread] [I] [toil.worker] Working on job 'minigraph_construct' kind-minigraph_construct/instance-wev60jsz v1
        [2024-09-05T18:46:28+1200] [MainThread] [I] [toil.worker] Loaded body Job('minigraph_construct' kind-minigraph_construct/instance-wev60jsz v1) from description 'minigraph_construct' kind-minigraph_construct/instance-wev60jsz v1
        [2024-09-05T18:46:28+1200] [MainThread] [C] [toil.worker] Worker crashed with traceback:
        Traceback (most recent call last):
          File "/home/cactus/cactus_env/lib/python3.10/site-packages/toil/worker.py", line 438, in workerScript
            job._runner(jobGraph=None, jobStore=job_store, fileStore=fileStore, defer=defer)
          File "/home/cactus/cactus_env/lib/python3.10/site-packages/toil/job.py", line 2984, in _runner
            returnValues = self._run(jobGraph=None, fileStore=fileStore)
          File "/home/cactus/cactus_env/lib/python3.10/site-packages/toil/job.py", line 2895, in _run
            return self.run(fileStore)
          File "/home/cactus/cactus_env/lib/python3.10/site-packages/toil/job.py", line 3158, in run
            rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs)
        TypeError: minigraph_construct() got an unexpected keyword argument 'has_resources'

        [2024-09-05T18:46:28+1200] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host
<=========

The running command is:

sudo singularity exec -B /mnt /mnt/container/cactus-v2.9.0.img cactus-pangenome ./js ./trio.seqfile \
 --outDir trio_pangenome \
 --workDir ./cactus-scratch \
 --outName trio \
 --reference GRCh38 \
 --vcfReference GRCh38 \
 --viz --odgi --vcfwave --vcf --chrom-og \
 --chrom-vg clip \
 --gbz clip full \
 --gfa clip full \
 --haplo \
 --filter 0 \
 --refContigs $(for i in `seq 22`; do printf "chr$i "; done ; echo "chrX chrY chrM") \
 --mgCores 60 \
 --mgMemory 300G \
 --mapCores 48 \
 --consCores 12 \
 --consMemory 300G \
 --indexCores 48 \
 --indexMemory 300G \
 --vcfwaveCores 48 \
 --vcfwaveMemory 150G \
 --defaultDisk 1200G \
 --defaultMemory 150G \
 --clusterStats ./clusterStats.json \
 --logFile ./trio-pangenome.log 2> trio_pangenome.stderr

Please let me know if you need additional information. Thank you once again for looking into this.

Regards,

glennhickey commented 2 months ago

Thanks for raising this, this indeed looks like a regression in 2.9.0. I think it only happens when using --mgMemory -- something apparently not covered in the tests. So if you don't specify this option, it should work fine. Will fix it in the next release.