search

ComparativeGenomicsToolkit / cactus

Official home of genome aligner based upon notion of Cactus graphs
Other
481 stars 106 forks source link

Job 'RedMaskJob' kind-PreprocessSequence/instance-rmupfpbr v8 is completely failed #1333

Closed song984888 closed 2 months ago

song984888 commented 3 months ago

Version: cactus-v2.8.0

Help?! what's wrong with my dataset. it is OK with exampleMammal data.

Enabling realtime logging in Toil Cactus Command: /panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/bin/cactus ./js my_run_good.txt ./myRun.hal --workDir ./tmp --stats --logFile ./log_1712026047 --clean never Cactus Commit: 7286b49b264896f43cc64aa405b39f914d43f75b No branch length for Juglans_californica: setting to 1

No branch length for Juglans_nigra: setting to 1

No branch length for : setting to 1

No branch length for Juglans_mandshurica: setting to 1

No branch length for : setting to 1

No branch length for Cyclocarya_paliurus: setting to 1

No branch length for Pterocarya_stenoptera: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Platycarya_strobilacea: setting to 1

No branch length for Platycarya_longipes: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Carya_cathayensis: setting to 1

No branch length for Carya_sinensis: setting to 1

No branch length for : setting to 1

No branch length for Carya_illinoinensis: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Engelhardia_specata: setting to 1

No branch length for Alfaropsis_roxburghiana: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Rhoiptelea_chiliantha: setting to 1

No branch length for : setting to 1

No branch length for Morella_rubrai: setting to 1

No branch length for : setting to 1

No branch length for Casuarina_equisetifolia: setting to 1

No branch length for Carpinus_fangiana: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Fagus_sylvatica: setting to 1

No branch length for Quercus_lobata: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Cucumis_sativus: setting to 1

No branch length for : setting to 1

No branch length for Fragaria_vesca: setting to 1

No branch length for : setting to 1

No branch length for Glycine_max: setting to 1

No branch length for : setting to 1

No branch length for Arabidopsis_thaliana: setting to 1

No branch length for : setting to 1

No branch length for Vitis_vinifera: setting to 1

No branch length for : setting to 1

No branch length for Solanum_lycopersicum: setting to 1

No branch length for Ambrosia_artemisiifolia: setting to 1

No branch length for : setting to 1

No branch length for : setting to 1

No branch length for Aquilegia_coerulea: setting to 1

No branch length for : setting to 1

No branch length for Acorus_gramineus: setting to 1

No branch length for : setting to 1

No branch length for Amborella_trichopoda: setting to 1

Tree: (((((((((((((((((((Juglans_californica:1.0,Juglans_nigra:1.0)Anc26:1.0,Juglans_mandshurica:1.0)Anc24:1.0,(Cyclocarya_paliurus:1.0,Pterocarya_stenoptera:1.0)Anc25:1.0)Anc21:1.0,(Platycarya_strobilacea:1.0,Platycarya_longipes:1.0)Anc22:1.0)Anc19:1.0,((Carya_cathayensis:1.0,Carya_sinensis:1.0)Anc23:1.0,Carya_illinoinensis:1.0)Anc20:1.0)Anc17:1.0,(Engelhardia_specata:1.0,Alfaropsis_roxburghiana:1.0)Anc18:1.0)Anc16:1.0,Rhoiptelea_chiliantha:1.0)Anc15:1.0,Morella_rubrai:1.0)Anc13:1.0,(Casuarina_equisetifolia:1.0,Carpinus_fangiana:1.0)Anc14:1.0)Anc11:1.0,(Fagus_sylvatica:1.0,Quercus_lobata:1.0)Anc12:1.0)Anc10:1.0,Cucumis_sativus:1.0)Anc09:1.0,Fragaria_vesca:1.0)Anc08:1.0,Glycine_max:1.0)Anc07:1.0,Arabidopsis_thaliana:1.0)Anc06:1.0,Vitis_vinifera:1.0)Anc04:1.0,(Solanum_lycopersicum:1.0,Ambrosia_artemisiifolia:1.0)Anc05:1.0)Anc03:1.0,Aquilegia_coerulea:1.0)Anc02:1.0,Acorus_gramineus:1.0)Anc01:1.0,Amborella_trichopoda:1.0)Anc00; Outgroups: Anc26: ['Juglans_mandshurica', 'Cyclocarya_paliurus'] Anc24: ['Anc25', 'Anc22'] Anc25: ['Juglans_mandshurica', 'Platycarya_strobilacea'] Anc21: ['Anc22', 'Anc20'] Anc22: ['Carya_illinoinensis', 'Juglans_mandshurica'] Anc19: ['Anc20', 'Anc18'] Anc23: ['Carya_illinoinensis', 'Rhoiptelea_chiliantha'] Anc20: ['Anc18', 'Anc22'] Anc17: ['Anc18', 'Rhoiptelea_chiliantha'] Anc18: ['Rhoiptelea_chiliantha', 'Morella_rubrai'] Anc16: ['Rhoiptelea_chiliantha', 'Morella_rubrai'] Anc15: ['Morella_rubrai', 'Anc14'] Anc13: ['Anc14', 'Anc12'] Anc14: ['Morella_rubrai', 'Cucumis_sativus'] Anc11: ['Anc12', 'Cucumis_sativus'] Anc12: ['Cucumis_sativus', 'Fragaria_vesca'] Anc10: ['Cucumis_sativus', 'Fragaria_vesca'] Anc09: ['Fragaria_vesca', 'Glycine_max'] Anc08: ['Glycine_max', 'Arabidopsis_thaliana'] Anc07: ['Arabidopsis_thaliana', 'Vitis_vinifera'] Anc06: ['Vitis_vinifera', 'Anc05'] Anc04: ['Anc05', 'Aquilegia_coerulea'] Anc05: ['Aquilegia_coerulea', 'Vitis_vinifera'] Anc03: ['Aquilegia_coerulea', 'Acorus_gramineus'] Anc02: ['Acorus_gramineus', 'Amborella_trichopoda'] Anc01: ['Amborella_trichopoda']

Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Aarte.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Acoer.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Agram.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Aroxb.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Athal.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Ccath.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Cequi.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Cfang.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Cpali.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Csati.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Csine.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Espec.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Fsylv.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Fvesc.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Gmax.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Jcali.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Jmand.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Jnigr.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Mrubr.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Plong.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Psten.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Pstro.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Qloba.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Rchil.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Slyco.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Vvini.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Cilli.fa.masked Importing file:///panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/genomes/Atric.fa.masked Saving graph of 1 jobs, 1 non-service, 1 new Processing job 'progressive_workflow' kind-progressive_workflow/instance-xb6tlw_s v0 Running Toil version 5.12.0-6d5a5b83b649cd8adf34a5cfe89e7690c95189d3 on host admin186. Starting real-time logging. Issued job 'progressive_workflow' kind-progressive_workflow/instance-xb6tlw_s v1 with job batch system ID: 1 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 1 jobs are running, 0 jobs are issued and waiting to run Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-1y7r2scc v1 with job batch system ID: 2 and disk: 3.6 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'sanitize_fasta_header' kind-sanitize_fasta_header/instance-yhl3gxo0 v1 with job batch system ID: 3 and disk: 5.1 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False

The job seems to have left a log file, indicating failure: 'RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v5 Log from job "kind-RedMaskJob/instance-tdwt8zxf" follows: =========> [2024-04-02T10:53:02+0800] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2024-04-02T10:53:02+0800] [MainThread] [I] [toil] Running Toil version 5.12.0-6d5a5b83b649cd8adf34a5cfe89e7690c95189d3 on host admin186. [2024-04-02T10:53:02+0800] [MainThread] [I] [toil.worker] Working on job 'RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v4 [2024-04-02T10:53:03+0800] [MainThread] [I] [toil.worker] Loaded body Job('RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v4) from description 'RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v4 [2024-04-02T10:53:03+0800] [MainThread] [I] [cactus.shared.common] Running the command ['cactus_softmask2hardmask', '-b', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max/Glycine_max.fa'] [2024-04-02T10:53:03+0800] [MainThread] [I] [toil-rt] 2024-04-02 10:53:03.103040: Running the command: "cactus_softmask2hardmask -b /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max/Glycine_max.fa" [2024-04-02T10:53:13+0800] [MainThread] [I] [toil-rt] 2024-04-02 10:53:13.042928: Successfully ran: "cactus_softmask2hardmask -b /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max/Glycine_max.fa" in 9.9207 seconds [2024-04-02T10:53:13+0800] [MainThread] [I] [toil-rt] 2024-04-02 10:53:13.357903: Running the command: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-out-Glycine_max"

This is Red (REpeat Detector) designed and developed by Hani Zakaria Girgis, PhD.

Version: 05/22/2015

Calculating the length, k, of the k-mer based on the input genome ... 
[2024-04-02T10:53:26+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files:
[2024-04-02T10:53:26+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-PreprocessSequence/instance-gh37s5gg/cleanup/file-dc42640cbc2a403980240a155930df7a/tmpaaumlbyc.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max/Glycine_max.fa'
Traceback (most recent call last):
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/toil/worker.py", line 403, in workerScript
    job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/cactus/shared/common.py", line 976, in _runner
    fileStore=fileStore, **kwargs)
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/toil/job.py", line 2774, in _runner
    returnValues = self._run(jobGraph=None, fileStore=fileStore)
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/toil/job.py", line 2691, in _run
    return self.run(fileStore)
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/cactus/preprocessor/redMasking.py", line 62, in run
    cactus_call(parameters=red_cmd)
  File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.8.0/venv-cactus-v2.8.0/lib/python3.7/site-packages/cactus/shared/common.py", line 908, in cactus_call
    raise RuntimeError("{}Command {} signaled {}: {}".format(sigill_msg, call, signal.Signals(-process.returncode).name, out))
RuntimeError: Command ['Red', '-gnm', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-in-Glycine_max', '-msk', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/3ff8/d85f/tmpkrewrtcy/red-out-Glycine_max'] signaled SIGABRT: stderr=terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)

[2024-04-02T10:53:26+0800] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host admin186

<========= Due to failure we are reducing the remaining try count of job 'RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v5 with ID kind-RedMaskJob/instance-tdwt8zxf to 0 2024-04-02 10:57:55.500368: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/83ec/db9f/tmpsdrob76m/Cucumis_sativus.input.masking.bed" in 22.0526 seconds Red masked 97940263 bp of Rhoiptelea_chiliantha, increasing masking from 40965979 to 138906242 Red masked 153574870 bp of Cucumis_sativus, increasing masking from 18464070 to 172038940 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-5cpg63qh v1 with job batch system ID: 311 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-6crlsn3l v1 with job batch system ID: 312 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-_54fcm99 v1 with job batch system ID: 313 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-yu0x_tbh v1 with job batch system ID: 314 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:00:47.230817: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/231d/861a/tmp1zmlyeyq/red-in-Juglans_nigra -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/231d/861a/tmp1zmlyeyq/red-out-Juglans_nigra" in 542.771 seconds 2024-04-02 11:00:47.235243: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/231d/861a/tmp1zmlyeyq/Juglans_nigra.input.masking.bed" 2024-04-02 11:01:02.778434: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/db0f/53b4/tmpdqk06v0z/red-in-Vitis_vinifera -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/db0f/53b4/tmpdqk06v0z/red-out-Vitis_vinifera" in 541.3041 seconds 2024-04-02 11:01:02.782947: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/db0f/53b4/tmpdqk06v0z/Vitis_vinifera.input.masking.bed" 2024-04-02 11:01:14.447660: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/bdf4/bf08/tmpdk9ofcio/red-in-Juglans_mandshurica -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/bdf4/bf08/tmpdk9ofcio/red-out-Juglans_mandshurica" in 556.24 seconds 2024-04-02 11:01:14.451485: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/bdf4/bf08/tmpdk9ofcio/Juglans_mandshurica.input.masking.bed" 2024-04-02 11:01:17.217391: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f0/f1a4/tmp32c6khmf/red-in-Juglans_californica -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f0/f1a4/tmp32c6khmf/red-out-Juglans_californica" in 559.0752 seconds 2024-04-02 11:01:17.223013: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f0/f1a4/tmp32c6khmf/Juglans_californica.input.masking.bed" 2024-04-02 11:01:20.081831: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/231d/861a/tmp1zmlyeyq/Juglans_nigra.input.masking.bed" in 32.8222 seconds Red masked 160465270 bp of Juglans_nigra, increasing masking from 94460104 to 254925374 2024-04-02 11:01:30.246627: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/db0f/53b4/tmpdqk06v0z/Vitis_vinifera.input.masking.bed" in 27.4515 seconds Issued job 'clean_if_different' kind-clean_if_different/instance-pr9_op_n v1 with job batch system ID: 315 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-csva122q v1 with job batch system ID: 316 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Red masked 186048522 bp of Vitis_vinifera, increasing masking from 29450602 to 215499124 Issued job 'clean_if_different' kind-clean_if_different/instance-a3hj3xhx v1 with job batch system ID: 317 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-00n_lt3o v1 with job batch system ID: 318 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:01:47.444329: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/abda/cfda/tmp_pxv7n46/red-in-Fagus_sylvatica -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/abda/cfda/tmp_pxv7n46/red-out-Fagus_sylvatica" in 585.5033 seconds 2024-04-02 11:01:47.448796: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/abda/cfda/tmp_pxv7n46/Fagus_sylvatica.input.masking.bed" 2024-04-02 11:01:48.344391: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/bdf4/bf08/tmpdk9ofcio/Juglans_mandshurica.input.masking.bed" in 33.8812 seconds 2024-04-02 11:01:49.107740: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f0/f1a4/tmp32c6khmf/Juglans_californica.input.masking.bed" in 31.8631 seconds Red masked 155344380 bp of Juglans_mandshurica, increasing masking from 101265911 to 256610291 Red masked 156132849 bp of Juglans_californica, increasing masking from 101755363 to 257888212 2024-04-02 11:01:55.993126: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/619d/dfb6/tmp8hmvm3da/red-in-Amborella_trichopoda -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/619d/dfb6/tmp8hmvm3da/red-out-Amborella_trichopoda" in 569.5063 seconds 2024-04-02 11:01:55.998504: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/619d/dfb6/tmp8hmvm3da/Amborella_trichopoda.input.masking.bed" Issued job 'clean_if_different' kind-clean_if_different/instance-audmx8je v1 with job batch system ID: 319 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-pcpfsktr v1 with job batch system ID: 320 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-2_es_y1r v1 with job batch system ID: 321 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-ong5dunu v1 with job batch system ID: 322 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:02:01.700622: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/ac40/f8fa/tmpwjrpc_sd/red-in-Pterocarya_stenoptera -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/ac40/f8fa/tmpwjrpc_sd/red-out-Pterocarya_stenoptera" in 591.777 seconds 2024-04-02 11:02:01.705746: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/ac40/f8fa/tmpwjrpc_sd/Pterocarya_stenoptera.input.masking.bed" 2024-04-02 11:02:17.241651: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/abda/cfda/tmp_pxv7n46/Fagus_sylvatica.input.masking.bed" in 29.7767 seconds Red masked 178570693 bp of Fagus_sylvatica, increasing masking from 90760253 to 269330946 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-fgy405ko v1 with job batch system ID: 323 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-naujvb9h v1 with job batch system ID: 324 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:02:29.922620: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/ac40/f8fa/tmpwjrpc_sd/Pterocarya_stenoptera.input.masking.bed" in 28.208 seconds Red masked 162622864 bp of Pterocarya_stenoptera, increasing masking from 98272807 to 260895671 2024-04-02 11:02:37.963496: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/619d/dfb6/tmp8hmvm3da/Amborella_trichopoda.input.masking.bed" in 41.9417 seconds 2024-04-02 11:02:39.406937: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e1bb/14af/tmpa6e3ppqw/red-in-Carya_sinensis -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e1bb/14af/tmpa6e3ppqw/red-out-Carya_sinensis" in 629.0174 seconds 2024-04-02 11:02:39.418027: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e1bb/14af/tmpa6e3ppqw/Carya_sinensis.input.masking.bed" Issued job 'clean_if_different' kind-clean_if_different/instance-vdgsh3bv v1 with job batch system ID: 325 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-e9ijnd8r v1 with job batch system ID: 326 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:02:43.413323: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/55ec/6f6e/tmpquftpvm5/red-in-Solanum_lycopersicum -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/55ec/6f6e/tmpquftpvm5/red-out-Solanum_lycopersicum" in 604.511 seconds 2024-04-02 11:02:43.417373: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/55ec/6f6e/tmpquftpvm5/Solanum_lycopersicum.input.masking.bed" Red masked 96254581 bp of Amborella_trichopoda, increasing masking from 403343772 to 499598353 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-p_aui_eu v1 with job batch system ID: 327 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-s3rq39lf v1 with job batch system ID: 328 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:02:55.428785: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f4ca/1383/tmpyjy3tvyu/red-in-Cyclocarya_paliurus -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f4ca/1383/tmpyjy3tvyu/red-out-Cyclocarya_paliurus" in 645.031 seconds 2024-04-02 11:02:55.432861: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f4ca/1383/tmpyjy3tvyu/Cyclocarya_paliurus.input.masking.bed" 2024-04-02 11:03:09.929227: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e1bb/14af/tmpa6e3ppqw/Carya_sinensis.input.masking.bed" in 30.5011 seconds 2024-04-02 11:03:15.428261: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/2ed3/12bb/tmpefcf4f7i/red-in-Platycarya_longipes -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/2ed3/12bb/tmpefcf4f7i/red-out-Platycarya_longipes" in 658.1223 seconds 2024-04-02 11:03:15.432048: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/2ed3/12bb/tmpefcf4f7i/Platycarya_longipes.input.masking.bed" Red masked 173534132 bp of Carya_sinensis, increasing masking from 120011998 to 293546130 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-2qqy0vd5 v1 with job batch system ID: 329 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-pbyby8tv v1 with job batch system ID: 330 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:03:28.073096: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/55ec/6f6e/tmpquftpvm5/Solanum_lycopersicum.input.masking.bed" in 44.645 seconds 2024-04-02 11:03:29.414718: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f4ca/1383/tmpyjy3tvyu/Cyclocarya_paliurus.input.masking.bed" in 33.9734 seconds 2024-04-02 11:03:29.941378: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/042c/b4d4/tmpwnjnrh3d/red-in-Platycarya_strobilacea -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/042c/b4d4/tmpwnjnrh3d/red-out-Platycarya_strobilacea" in 664.5941 seconds 2024-04-02 11:03:29.946162: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/042c/b4d4/tmpwnjnrh3d/Platycarya_strobilacea.input.masking.bed" Red masked 183598084 bp of Cyclocarya_paliurus, increasing masking from 107686012 to 291284096 Red masked 112646025 bp of Solanum_lycopersicum, increasing masking from 548639696 to 661285721 2024-04-02 11:03:41.867999: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/159c/7ff5/tmpshqkzbxe/red-in-Carya_cathayensis -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/159c/7ff5/tmpshqkzbxe/red-out-Carya_cathayensis" in 679.5068 seconds 2024-04-02 11:03:41.871574: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/159c/7ff5/tmpshqkzbxe/Carya_cathayensis.input.masking.bed" Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-dc577ejt v1 with job batch system ID: 331 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-wnckyvk9 v1 with job batch system ID: 332 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'clean_if_different' kind-clean_if_different/instance-xaxvyet7 v1 with job batch system ID: 333 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-0sbaull0 v1 with job batch system ID: 334 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:03:50.059574: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/2ed3/12bb/tmpefcf4f7i/Platycarya_longipes.input.masking.bed" in 34.6176 seconds Red masked 188138724 bp of Platycarya_longipes, increasing masking from 138022627 to 326161351 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-1qzo8pei v1 with job batch system ID: 335 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-38vc_ut7 v1 with job batch system ID: 336 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:04:05.878589: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/042c/b4d4/tmpwnjnrh3d/Platycarya_strobilacea.input.masking.bed" in 35.9251 seconds Red masked 182032853 bp of Platycarya_strobilacea, increasing masking from 165082132 to 347114985 2024-04-02 11:04:16.861245: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/159c/7ff5/tmpshqkzbxe/Carya_cathayensis.input.masking.bed" in 34.9798 seconds 2024-04-02 11:04:17.475141: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/9a1b/cde4/tmppn2cv5sj/red-in-Engelhardia_specata -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/9a1b/cde4/tmppn2cv5sj/red-out-Engelhardia_specata" in 703.9383 seconds 2024-04-02 11:04:17.521169: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/9a1b/cde4/tmppn2cv5sj/Engelhardia_specata.input.masking.bed" Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-ok3ib2zm v1 with job batch system ID: 337 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-9xxxruaz v1 with job batch system ID: 338 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Red masked 208069804 bp of Carya_cathayensis, increasing masking from 137768268 to 345838072 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-1bdysv_m v1 with job batch system ID: 339 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-ytdtanap v1 with job batch system ID: 340 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:04:35.054232: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/552e/7de4/tmpxo1mdjps/red-in-Acorus_gramineus -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/552e/7de4/tmpxo1mdjps/red-out-Acorus_gramineus" in 713.9462 seconds 2024-04-02 11:04:35.059136: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/552e/7de4/tmpxo1mdjps/Acorus_gramineus.input.masking.bed" 2024-04-02 11:04:53.970044: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/9a1b/cde4/tmppn2cv5sj/Engelhardia_specata.input.masking.bed" in 36.4414 seconds Red masked 239198392 bp of Engelhardia_specata, increasing masking from 189639896 to 428838288 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-rjt6dm2t v1 with job batch system ID: 341 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-4i5e7mn7 v1 with job batch system ID: 342 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:05:13.334166: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/43be/fb43/tmpjayn2dra/red-in-Alfaropsis_roxburghiana -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/43be/fb43/tmpjayn2dra/red-out-Alfaropsis_roxburghiana" in 747.2575 seconds 2024-04-02 11:05:13.339735: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/43be/fb43/tmpjayn2dra/Alfaropsis_roxburghiana.input.masking.bed" 2024-04-02 11:05:14.945083: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/552e/7de4/tmpxo1mdjps/Acorus_gramineus.input.masking.bed" in 39.8769 seconds Red masked 317256844 bp of Acorus_gramineus, increasing masking from 95074880 to 412331724 2024-04-02 11:05:26.138733: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f0f0/2ddd/tmpgqjnn2gh/red-in-Quercus_lobata -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f0f0/2ddd/tmpgqjnn2gh/red-out-Quercus_lobata" in 771.838 seconds 2024-04-02 11:05:26.152715: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f0f0/2ddd/tmpgqjnn2gh/Quercus_lobata.input.masking.bed" Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-gskp8gd9 v1 with job batch system ID: 343 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-rf_9a0a9 v1 with job batch system ID: 344 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-02 11:05:55.306187: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/43be/fb43/tmpjayn2dra/Alfaropsis_roxburghiana.input.masking.bed" in 41.958 seconds Red masked 281036903 bp of Alfaropsis_roxburghiana, increasing masking from 185407578 to 466444481 2024-04-02 11:06:05.380823: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/f0f0/2ddd/tmpgqjnn2gh/Quercus_lobata.input.masking.bed" in 39.1759 seconds Issued job 'clean_if_different' kind-clean_if_different/instance-elz9rd_z v1 with job batch system ID: 345 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-jzk1xv79 v1 with job batch system ID: 346 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Red masked 279178025 bp of Quercus_lobata, increasing masking from 122807477 to 401985502 Issued job 'clean_if_different' kind-clean_if_different/instance-mrfrs089 v1 with job batch system ID: 347 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-qb4k5y8n v1 with job batch system ID: 348 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:07:02.397601: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e2f9/0f86/tmpz0byno1e/red-in-Ambrosia_artemisiifolia -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e2f9/0f86/tmpz0byno1e/red-out-Ambrosia_artemisiifolia" in 858.8958 seconds 2024-04-02 11:07:02.402977: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e2f9/0f86/tmpz0byno1e/Ambrosia_artemisiifolia.input.masking.bed" 2024-04-02 11:07:45.305536: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/e2f9/0f86/tmpz0byno1e/Ambrosia_artemisiifolia.input.masking.bed" in 42.8942 seconds Red masked 420397129 bp of Ambrosia_artemisiifolia, increasing masking from 145950656 to 566347785 Issued job 'clean_if_different' kind-clean_if_different/instance-sah76d5f v1 with job batch system ID: 349 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-7sgu3z9r v1 with job batch system ID: 350 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True 2024-04-02 11:08:55.501224: Successfully ran: "Red -gnm /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f5/d022/tmptp8z4q01/red-in-Carya_illinoinensis -msk /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f5/d022/tmptp8z4q01/red-out-Carya_illinoinensis" in 920.8014 seconds 2024-04-02 11:08:55.505564: Running the command: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f5/d022/tmptp8z4q01/Carya_illinoinensis.input.masking.bed" 2024-04-02 11:09:53.934579: Successfully ran: "cactus_fasta_softmask_intervals.py --origin=zero /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmp/de5a36d5b7df53a19dd2971439441afc/71f5/d022/tmptp8z4q01/Carya_illinoinensis.input.masking.bed" in 58.4188 seconds Red masked 416261591 bp of Carya_illinoinensis, increasing masking from 287689988 to 703951579 Issued job 'BatchPreprocessor' kind-BatchPreprocessor/instance-1j51clpe v1 with job batch system ID: 351 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: True Issued job 'clean_if_different' kind-clean_if_different/instance-t_ylwy4u v1 with job batch system ID: 352 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Finished toil run with 8 failed jobs. Failed jobs at end of the run: 'BatchPreprocessor' kind-CactusPreprocessor2/instance-t6ca0q2p v4 'CactusPreprocessor' kind-CactusPreprocessor/instance-rbazmd4c v2 'BatchPreprocessor' kind-BatchPreprocessor/instance-anbimtwy v3 'sanitize_fasta_headers' kind-progressive_workflow/instance-xb6tlw_s v4 'Job' kind-preprocess_all/instance-_p0zum04 v5 'RedMaskJob' kind-RedMaskJob/instance-tdwt8zxf v6 'PreprocessSequence' kind-PreprocessSequence/instance-gh37s5gg v2 'BatchPreprocessor' kind-BatchPreprocessor/instance-ksz3sjc7 v3 Stopping real-time logging server. Joining real-time logging server thread.

glennhickey commented 3 months ago

This is a known issue that will be fixed in the next Cactus release (hopefully sometime this week).

Your simplest work-around in the meantime is probably just to use the previous version of Cactus.

song984888 commented 2 months ago

This is a known issue that will be fixed in the next Cactus release (hopefully sometime this week).

Your simplest work-around in the meantime is probably just to use the previous version of Cactus.

Thanks for your suggestion. I re-run the same dataset using cactus-2.7.0, but it still failed.

...... 2024-04-03 23:51:21.692194: Successfully ran: "paffy trim -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/tiled_Anc20.paf --trimIdentity 0.2" in 0.0004 seconds 2024-04-03 23:51:21.694862: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/trim_Anc20.paf --maxTileLevel 1" 2024-04-03 23:51:21.699571: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/trim_Anc20.paf --maxTileLevel 1" in 0.0003 seconds 2024-04-03 23:51:21.702359: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/filter_Anc20.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:21.706927: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/filter_Anc20.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0004 seconds 2024-04-03 23:51:21.709544: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/primary_chain_Anc20.paf --minChainScore 10000" 2024-04-03 23:51:21.714154: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/035c/01f5/tmpi44dqajc/primary_chain_Anc20.paf --minChainScore 10000" in 0.0003 seconds Got message from job at time 04-03-2024 23:51:21: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-trim_unaligned_sequences/instance-boj_gk33/cleanup/file-aa3237ddf91d4345b0a3983d03daf85a/stream used 0.00% disk (400.0 KiB [409600B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:21: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-tile_alignments/instance-83syzfpe/cleanup/file-dc9b336f2b0f45098d8f6c6594a6e347/stream used 0.00% disk (224.0 KiB [229376B] used, 0 bB [0B] requested). Estimating cactus_consolidated(Anc24) memory as 3.7 Gi from 4 sequences with total-sequence-size 9.8 Ki and paf-size 0 using configuration settings Issued job 'merge_alignments' kind-merge_alignments/instance-0590b4y9 v1 with job batch system ID: 576 and disk: 0, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'cactus_cons' kind-cactus_cons/instance-t4adhxyj v1 with job batch system ID: 577 and disk: 49.0 Ki, memory: 3.7 Gi, cores: 40, accelerators: [], preemptible: False 2024-04-03 23:51:23.790463: Running the command: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/f536/b125/tmpyawqiy2d/alignment_1.paf /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/f536/b125/tmpyawqiy2d/alignment_2.paf" 2024-04-03 23:51:23.795813: Successfully ran: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/f536/b125/tmpyawqiy2d/alignment_1.paf /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/f536/b125/tmpyawqiy2d/alignment_2.paf" in 0.0013 seconds with job-memory 2.0 Gi Got message from job at time 04-03-2024 23:51:23: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-merge_alignments/instance-0590b4y9/cleanup/file-d63f71fffab444d892db91698ce9a6c6/stream used 0.00% disk (272.0 KiB [278528B] used, 0 bB [0B] requested). 2024-04-03 23:51:24.811575: Running the command: "cactus_consolidated --sequences Anc26 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc26.fa Juglans_mandshurica /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Juglans_mandshurica.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc22.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc25.fa --speciesTree (((Anc26:1.0,Juglans_mandshurica:1.0)Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55ca <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.ref --outgroupEvents Anc25 Anc22 --referenceEvent Anc24 --threads 40" cactus_consolidated(Anc24): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.config.xml cactus_consolidated(Anc24): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.c2h cactus_consolidated(Anc24): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.c2h.fa cactus_consolidated(Anc24): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.ref cactus_consolidated(Anc24): Sequence files and events: Anc26 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc26.fa Juglans_mandshurica /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Juglans_mandshurica.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc22.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc25.fa cactus_consolidated(Anc24): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24_primary.paf cactus_consolidated(Anc24): Secondary alignments file: (null) cactus_consolidated(Anc24): Constraint alignments file: (null) cactus_consolidated(Anc24): Species tree: (((Anc26:1.0,Juglans_mandshurica:1.0)Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19; cactus_consolidated(Anc24): Outgroup events: Anc25 Anc22 cactus_consolidated(Anc24): Reference event: Anc24 cactus_consolidated(Anc24): Loaded the parameters files, 0 seconds have elapsed cactus_consolidated(Anc24): Set up the cactus disk, 0 seconds have elapsed cactus_consolidated(Anc24): Constructed the first flower cactus_consolidated(Anc24): Going to build the event tree with newick string: (((Anc26:1.0,Juglans_mandshurica:1.0)Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19; cactus_consolidated(Anc24): Parsed the tree cactus_consolidated(Anc24): Constructed the basic event tree cactus_consolidated(Anc24): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc26.fa to Anc26 cactus_consolidated(Anc24): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc26.fa cactus_consolidated(Anc24): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Juglans_mandshurica.fa to Juglans_mandshurica cactus_consolidated(Anc24): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Juglans_mandshurica.fa cactus_consolidated(Anc24): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc22.fa to Anc22 cactus_consolidated(Anc24): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc22.fa cactus_consolidated(Anc24): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc25.fa to Anc25 cactus_consolidated(Anc24): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc25.fa cactus_consolidated(Anc24): Constructed the initial flower with 1 sequences and 8 events with string: ((((Anc26:1,Juglans_mandshurica:1)Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:9.22337e+18)ROOT:9.22337e+18; cactus_consolidated(Anc24): Established the first Flower in the hierarchy, 0 seconds have elapsed cactus_consolidated(Anc24): Converted alignment coordinates, 0 seconds have elapsed cactus_consolidated(Anc24): Stripped any unique IDs, 0 seconds have elapsed cactus_consolidated(Anc24): We found a max path distance between ingroups in the tree (((((Anc26:1,Juglans_mandshurica:1)Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:9.22337e+18)ROOT:9.22337e+18;) of 2.000000, giving us and min final chain length of: 256 cactus_consolidated(Anc24): Starting annealing round with a minimum chain length of 256 and an alignment trim of 3 cactus_consolidated(Anc24): Sequence graph statistics after annealing: cactus_consolidated(Anc24): There were 2 blocks in the sequence graph, representing 2 total aligned bases cactus_consolidated(Anc24): Block degree stats: min 1, avg 1.000000, median 1, max 1 cactus_consolidated(Anc24): Block support stats: min 0.000000, avg 0.000000, median 0.000000, max 0.000000 cactus_consolidated(Anc24): Starting melting round with a minimum chain length of 2 cactus_consolidated(Anc24): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 2. Total aligned bases lost: 0 cactus_consolidated(Anc24): Starting melting round with a minimum chain length of 32 cactus_consolidated(Anc24): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 32. Total aligned bases lost: 0 cactus_consolidated(Anc24): Starting melting round with a minimum chain length of 256 cactus_consolidated(Anc24): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 256. Total aligned bases lost: 0 cactus_consolidated(Anc24): Destroying 0 recoverable blocks cactus_consolidated(Anc24): The blocks covered 0 columns for a total of 0 aligned bases cactus_consolidated(Anc24): Sequence graph statistics after melting: cactus_consolidated(Anc24): There were 2 blocks in the sequence graph, representing 2 total aligned bases cactus_consolidated(Anc24): Block degree stats: min 1, avg 1.000000, median 1, max 1 cactus_consolidated(Anc24): Block support stats: min 0.000000, avg 0.000000, median 0.000000, max 0.000000 cactus_consolidated(Anc24): Ran cactus caf, 0 seconds have elapsed cactus_consolidated(Anc24): Ran extended flowers ready for bar, 0 seconds have elapsed cactus_consolidated(Anc24): Ran cactus bar (use poa:1), 0 seconds have elapsed cactus_consolidated(Anc24): There are 2 layers in the flowers hierarchy cactus_consolidated(Anc24): In the 0 layer there are 1 flowers in the flowers hierarchy cactus_consolidated(Anc24): Chose reference event 4: Anc24 cactus_consolidated(Anc24): For flower: 0 we have 0 nodes for: 2 ends, 1 chains, 0 stubs and 0 blocks cactus_consolidated(Anc24): Building a matching for 0 stub nodes in the top level problem from 2 total stubs of which 2 attached , 2 total ends, 1 chains, 0 blocks 1 groups and 1 sequences cactus_consolidated(Anc24): Starting to build the reference for flower 0, with 0 stubs and 0 chains and 0 nodes in the flowers tangle cactus_consolidated(Anc24): The score of the initial solution is 0.000000/0 out of a max possible 0.000000 cactus_consolidated(Anc24): The score of the solution after permutation sampling is 0.000000/0 after 10 rounds of greedy permutation out of a max possible 0.000000 cactus_consolidated(Anc24): The score of the final reference solution is 0.000000/0 after 100 rounds of greedy nudging out of a max possible 0.000000 cactus_consolidated(Anc24): In the 1 layer there are 1 flowers in the flowers hierarchy cactus_consolidated(Anc24): Ran cactus make reference, 0 seconds have elapsed cactus_consolidated(Anc24): Ran cactus make reference bottom up coordinates, 0 seconds have elapsed cactus_consolidated(Anc24): Ran cactus make reference top down coordinates, 0 seconds have elapsed cactus_consolidated(Anc24): Ran cactus to hal stage, 0 seconds have elapsed cactus_consolidated(Anc24): Dumped sequences for hal file, 0 seconds have elapsed cactus_consolidated(Anc24): Dumped reference sequences, 0 seconds have elapsed cactus_consolidated(Anc24): Cactus consolidated is done!, 0 seconds have elapsed 2024-04-03 23:51:24.867491: Successfully ran cactus_consolidated(Anc24): "cactus_consolidated --sequences Anc26 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc26.fa Juglans_mandshurica /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Juglans_mandshurica.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc22.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc25.fa --speciesTree (((Anc26:1.0,Juglans_mandshurica:1.0)Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus <...> /Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/ebab/8199/tmpa1yk2vvo/Anc24.ref --outgroupEvents Anc25 Anc22 --referenceEvent Anc24 --threads 40" in 0.0489 seconds with job-memory 3.7 Gi Got message from job at time 04-03-2024 23:51:25: Ran cactus consolidated okay Got message from job at time 04-03-2024 23:51:25: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-cactus_cons/instance-t4adhxyj/cleanup/file-45f9b8a07d7a44d1bdfb7077551efe54/stream used 1240.59% disk (608.0 KiB [622592B] used, 49.0 KiB [50185B] requested). Issued job 'Job' kind-Job/instance-d0osu0gi v1 with job batch system ID: 578 and disk: 95.4 Mi, memory: 488.3 Mi, cores: 0.1, accelerators: [], preemptible: False Issued job 'trim_unaligned_sequences' kind-trim_unaligned_sequences/instance-x_59o26k v1 with job batch system ID: 579 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:28.728528: Running the command: "paffy to_bed --binary --excludeUnaligned --includeInverted -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmprx3h332x.tmp --logLevel INFO" 2024-04-03 23:51:28.733180: Successfully ran: "paffy to_bed --binary --excludeUnaligned --includeInverted -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmprx3h332x.tmp --logLevel INFO" in 0.0008 seconds 2024-04-03 23:51:28.740675: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp0t0_pgl3.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp5zy_f29d.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" 2024-04-03 23:51:28.750106: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp0t0_pgl3.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp5zy_f29d.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" in 0.0009 seconds 2024-04-03 23:51:28.757793: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp0t0_pgl3.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp3pwlcqoq.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" 2024-04-03 23:51:28.763193: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp0t0_pgl3.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmp3pwlcqoq.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" in 0.0008 seconds 2024-04-03 23:51:28.766907: Running the command: "paffy upconvert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmprx3h332x.tmp --logLevel INFO /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmpjlbzrxkj.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmplt4n7ha2.tmp" 2024-04-03 23:51:28.771784: Successfully ran: "paffy upconvert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmprx3h332x.tmp --logLevel INFO /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmpjlbzrxkj.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/369f/c5f8/tmplt4n7ha2.tmp" in 0.0004 seconds Issued job 'progressive_step_2' kind-progressive_step_2/instance-8j2s7h3k v1 with job batch system ID: 580 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Got message from job at time 04-03-2024 23:51:29: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-trim_unaligned_sequences/instance-x_59o26k/cleanup/file-a687a1c1e9d348d5b8a117e59d8e18a1/stream used 0.00% disk (400.0 KiB [409600B] used, 0 bB [0B] requested). Estimating cactus_consolidated(Anc20) memory as 3.7 Gi from 4 sequences with total-sequence-size 9.8 Ki and paf-size 0 using configuration settings Issued job 'cactus_cons' kind-cactus_cons/instance-kh6gn590 v1 with job batch system ID: 581 and disk: 49.0 Ki, memory: 3.7 Gi, cores: 40, accelerators: [], preemptible: False 2024-04-03 23:51:31.238587: Running the command: "cactus_consolidated --sequences Anc23 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc23.fa Carya_illinoinensis /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Carya_illinoinensis.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc22.fa Anc18 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc18.fa --speciesTree ((Anc22:2.0,(Anc23:1.0,Carya_illinoinensis:1.0)Anc20:1.0)Anc17:1.0,Anc18:1.0)Anc16; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55ca <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.ref --outgroupEvents Anc18 Anc22 --referenceEvent Anc20 --threads 40" cactus_consolidated(Anc20): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.config.xml cactus_consolidated(Anc20): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.c2h cactus_consolidated(Anc20): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.c2h.fa cactus_consolidated(Anc20): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.ref cactus_consolidated(Anc20): Sequence files and events: Anc23 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc23.fa Carya_illinoinensis /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Carya_illinoinensis.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc22.fa Anc18 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc18.fa cactus_consolidated(Anc20): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20_primary.paf cactus_consolidated(Anc20): Secondary alignments file: (null) cactus_consolidated(Anc20): Constraint alignments file: (null) cactus_consolidated(Anc20): Species tree: ((Anc22:2.0,(Anc23:1.0,Carya_illinoinensis:1.0)Anc20:1.0)Anc17:1.0,Anc18:1.0)Anc16; cactus_consolidated(Anc20): Outgroup events: Anc18 Anc22 cactus_consolidated(Anc20): Reference event: Anc20 cactus_consolidated(Anc20): Loaded the parameters files, 0 seconds have elapsed cactus_consolidated(Anc20): Set up the cactus disk, 0 seconds have elapsed cactus_consolidated(Anc20): Constructed the first flower cactus_consolidated(Anc20): Going to build the event tree with newick string: ((Anc22:2.0,(Anc23:1.0,Carya_illinoinensis:1.0)Anc20:1.0)Anc17:1.0,Anc18:1.0)Anc16; cactus_consolidated(Anc20): Parsed the tree cactus_consolidated(Anc20): Constructed the basic event tree cactus_consolidated(Anc20): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc23.fa to Anc23 cactus_consolidated(Anc20): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc23.fa cactus_consolidated(Anc20): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Carya_illinoinensis.fa to Carya_illinoinensis cactus_consolidated(Anc20): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Carya_illinoinensis.fa cactus_consolidated(Anc20): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc22.fa to Anc22 cactus_consolidated(Anc20): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc22.fa cactus_consolidated(Anc20): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc18.fa to Anc18 cactus_consolidated(Anc20): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc18.fa cactus_consolidated(Anc20): Constructed the initial flower with 1 sequences and 8 events with string: (((Anc22:2,(Anc23:1,Carya_illinoinensis:1)Anc20:1)Anc17:1,Anc18:1)Anc16:9.22337e+18)ROOT:9.22337e+18; cactus_consolidated(Anc20): Established the first Flower in the hierarchy, 0 seconds have elapsed cactus_consolidated(Anc20): Converted alignment coordinates, 0 seconds have elapsed cactus_consolidated(Anc20): Stripped any unique IDs, 0 seconds have elapsed cactus_consolidated(Anc20): We found a max path distance between ingroups in the tree ((((Anc22:2,(Anc23:1,Carya_illinoinensis:1)Anc20:1)Anc17:1,Anc18:1)Anc16:9.22337e+18)ROOT:9.22337e+18;) of 2.000000, giving us and min final chain length of: 256 cactus_consolidated(Anc20): Starting annealing round with a minimum chain length of 256 and an alignment trim of 3 cactus_consolidated(Anc20): Sequence graph statistics after annealing: cactus_consolidated(Anc20): There were 2 blocks in the sequence graph, representing 2 total aligned bases cactus_consolidated(Anc20): Block degree stats: min 1, avg 1.000000, median 1, max 1 cactus_consolidated(Anc20): Block support stats: min 0.000000, avg 0.000000, median 0.000000, max 0.000000 cactus_consolidated(Anc20): Starting melting round with a minimum chain length of 2 cactus_consolidated(Anc20): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 2. Total aligned bases lost: 0 cactus_consolidated(Anc20): Starting melting round with a minimum chain length of 32 cactus_consolidated(Anc20): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 32. Total aligned bases lost: 0 cactus_consolidated(Anc20): Starting melting round with a minimum chain length of 256 cactus_consolidated(Anc20): A melting round is destroying 0 blocks with an average degree of 0.000000 from chains with length less than 256. Total aligned bases lost: 0 cactus_consolidated(Anc20): Destroying 0 recoverable blocks cactus_consolidated(Anc20): The blocks covered 0 columns for a total of 0 aligned bases cactus_consolidated(Anc20): Sequence graph statistics after melting: cactus_consolidated(Anc20): There were 2 blocks in the sequence graph, representing 2 total aligned bases cactus_consolidated(Anc20): Block degree stats: min 1, avg 1.000000, median 1, max 1 cactus_consolidated(Anc20): Block support stats: min 0.000000, avg 0.000000, median 0.000000, max 0.000000 cactus_consolidated(Anc20): Ran cactus caf, 0 seconds have elapsed cactus_consolidated(Anc20): Ran extended flowers ready for bar, 0 seconds have elapsed cactus_consolidated(Anc20): Ran cactus bar (use poa:1), 0 seconds have elapsed cactus_consolidated(Anc20): There are 2 layers in the flowers hierarchy cactus_consolidated(Anc20): In the 0 layer there are 1 flowers in the flowers hierarchy cactus_consolidated(Anc20): Chose reference event 5: Anc20 cactus_consolidated(Anc20): For flower: 0 we have 0 nodes for: 2 ends, 1 chains, 0 stubs and 0 blocks cactus_consolidated(Anc20): Building a matching for 0 stub nodes in the top level problem from 2 total stubs of which 2 attached , 2 total ends, 1 chains, 0 blocks 1 groups and 1 sequences cactus_consolidated(Anc20): Starting to build the reference for flower 0, with 0 stubs and 0 chains and 0 nodes in the flowers tangle cactus_consolidated(Anc20): The score of the initial solution is 0.000000/0 out of a max possible 0.000000 cactus_consolidated(Anc20): The score of the solution after permutation sampling is 0.000000/0 after 10 rounds of greedy permutation out of a max possible 0.000000 cactus_consolidated(Anc20): The score of the final reference solution is 0.000000/0 after 100 rounds of greedy nudging out of a max possible 0.000000 cactus_consolidated(Anc20): In the 1 layer there are 1 flowers in the flowers hierarchy cactus_consolidated(Anc20): Ran cactus make reference, 0 seconds have elapsed cactus_consolidated(Anc20): Ran cactus make reference bottom up coordinates, 0 seconds have elapsed cactus_consolidated(Anc20): Ran cactus make reference top down coordinates, 0 seconds have elapsed cactus_consolidated(Anc20): Ran cactus to hal stage, 0 seconds have elapsed cactus_consolidated(Anc20): Dumped sequences for hal file, 0 seconds have elapsed cactus_consolidated(Anc20): Dumped reference sequences, 0 seconds have elapsed cactus_consolidated(Anc20): Cactus consolidated is done!, 0 seconds have elapsed 2024-04-03 23:51:31.296147: Successfully ran cactus_consolidated(Anc20): "cactus_consolidated --sequences Anc23 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc23.fa Carya_illinoinensis /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Carya_illinoinensis.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc22.fa Anc18 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc18.fa --speciesTree ((Anc22:2.0,(Anc23:1.0,Carya_illinoinensis:1.0)Anc20:1.0)Anc17:1.0,Anc18:1.0)Anc16; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus <...> /Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/9f5b/6532/tmpchjcumo4/Anc20.ref --outgroupEvents Anc18 Anc22 --referenceEvent Anc20 --threads 40" in 0.0513 seconds with job-memory 3.7 Gi Got message from job at time 04-03-2024 23:51:31: Ran cactus consolidated okay Got message from job at time 04-03-2024 23:51:31: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-cactus_cons/instance-kh6gn590/cleanup/file-b0295613c86f42778d4c25715a934edd/stream used 1239.85% disk (608.0 KiB [622592B] used, 49.0 KiB [50215B] requested). Issued job 'progressive_step' kind-progressive_step/instance-1t757117 v1 with job batch system ID: 582 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'EncapsulatedJob' kind-EncapsulatedJob/instance-gjs3eyi9 v1 with job batch system ID: 583 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_paf_alignments' kind-make_paf_alignments/instance-i_54lo65 v1 with job batch system ID: 584 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_ingroup_to_outgroup_alignments_0' kind-make_ingroup_to_outgroup_alignments_0/instance-r7h5uj2o v1 with job batch system ID: 585 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_ingroup_to_outgroup_alignments_0' kind-make_ingroup_to_outgroup_alignments_0/instance-t_tq2fmq v1 with job batch system ID: 586 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_chunked_alignments' kind-make_chunked_alignments/instance-6_fkwk62 v1 with job batch system ID: 587 and disk: 0, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:42.121879: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpmef0lo9d /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpda4nnw36.tmp" 2024-04-03 23:51:42.128818: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpmef0lo9d /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpda4nnw36.tmp" in 0.0018 seconds 2024-04-03 23:51:42.136466: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpnleoadxq /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpybu2ngd8.tmp" 2024-04-03 23:51:42.142421: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpnleoadxq /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2456/0e3a/tmpybu2ngd8.tmp" in 0.0014 seconds Issued job 'combine_chunks' kind-combine_chunks/instance-9bc9xc09 v1 with job batch system ID: 588 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:42.348049: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmpqvxt1ih1 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmp1cp4liaa.tmp" 2024-04-03 23:51:42.354204: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmpqvxt1ih1 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmp1cp4liaa.tmp" in 0.0014 seconds 2024-04-03 23:51:42.361171: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmpcfkldd58 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmprxew4g4m.tmp" 2024-04-03 23:51:42.361288: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmpfaqki773 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmpthntkc0o.tmp" 2024-04-03 23:51:42.366795: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmpcfkldd58 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/7786/de5e/tmprxew4g4m.tmp" in 0.0013 seconds 2024-04-03 23:51:42.366936: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmpfaqki773 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmpthntkc0o.tmp" in 0.0014 seconds 2024-04-03 23:51:42.374262: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmp989xc6vc /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmp1fk4je4z.tmp" 2024-04-03 23:51:42.379675: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmp989xc6vc /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/378a/d6d0/tmp1fk4je4z.tmp" in 0.0013 seconds Got message from job at time 04-03-2024 23:51:42: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_chunked_alignments/instance-6_fkwk62/cleanup/file-3ef1db6db58642cbb81d792c262c6221/stream used 0.00% disk (288.0 KiB [294912B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:42: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_chunked_alignments/instance-r6fke6jh/cleanup/file-64b527b866074a9788bd83e23db12186/stream used 0.00% disk (288.0 KiB [294912B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:42: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_chunked_alignments/instance-ccsxweo3/cleanup/file-d89a08599f8f4912b9648654bb0d14a7/stream used 0.00% disk (288.0 KiB [294912B] used, 0 bB [0B] requested). Issued job 'combine_chunks' kind-combine_chunks/instance-v9a1b3rc v1 with job batch system ID: 589 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'combine_chunks' kind-combine_chunks/instance-5edjrx12 v1 with job batch system ID: 590 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_ingroup_to_outgroup_alignments_2' kind-make_ingroup_to_outgroup_alignments_2/instance-dgsxegyx v1 with job batch system ID: 591 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False Issued job 'make_ingroup_to_outgroup_alignments_2' kind-make_ingroup_to_outgroup_alignments_2/instance-5f1glrv8 v1 with job batch system ID: 592 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:44.600153: Running the command: "paffy to_bed --excludeAligned --binary --minSize 100 -i /panfs4/gpu/home/songht/plantsgenomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmpin22ukh.tmp --logLevel INFO" 2024-04-03 23:51:44.606481: Successfully ran: "paffy to_bed --excludeAligned --binary --minSize 100 -i /panfs4/gpu/home/songht/plantsgenomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmpin22ukh.tmp --logLevel INFO" in 0.0012 seconds 2024-04-03 23:51:44.607057: Running the command: "paffy to_bed --excludeAligned --binary --minSize 100 -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmpencvwsc2.tmp --logLevel INFO" 2024-04-03 23:51:44.613071: Successfully ran: "paffy to_bed --excludeAligned --binary --minSize 100 -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmpencvwsc2.tmp --logLevel INFO" in 0.0009 seconds 2024-04-03 23:51:44.614044: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmpi2qmc2v6.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmp4t50dt_d.tmp --flank 100 --logLevel INFO" 2024-04-03 23:51:44.620573: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmpi2qmc2v6.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2f3e/a475/tmp4t50dt_d.tmp --flank 100 --logLevel INFO" in 0.0009 seconds 2024-04-03 23:51:44.621134: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmph0q3t08c.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmp5u8wg5ll.tmp --flank 100 --logLevel INFO" 2024-04-03 23:51:44.626935: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmph0q3t08c.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/60e0/7049/tmp5u8wg5ll.tmp --flank 100 --logLevel INFO" in 0.0009 seconds Issued job 'Job' kind-Job/instance-rq1klotw v1 with job batch system ID: 593 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'Job' kind-Job/instance-4lfftezx v1 with job batch system ID: 594 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Got message from job at time 04-03-2024 23:51:45: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_ingroup_to_outgroup_alignments_2/instance-dgsxegyx/cleanup/file-7a1d12ccd3224b6795302a76152e1a33/stream used 0.00% disk (256.0 KiB [262144B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:45: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_ingroup_to_outgroup_alignments_2/instance-5f1glrv8/cleanup/file-46addc61e9b74676b06a7ebdb0a5d8de/stream used 0.00% disk (256.0 KiB [262144B] used, 0 bB [0B] requested). 2024-04-03 23:51:45.951442: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmp2maq8997 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpcxc03rj2.tmp" 2024-04-03 23:51:45.958480: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmp2maq8997 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpcxc03rj2.tmp" in 0.002 seconds 2024-04-03 23:51:45.966730: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpyozvude0 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpu2nhzeiv.tmp" 2024-04-03 23:51:45.971895: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plantsgenomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmpurrgc9c /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmp1sznlek9.tmp" 2024-04-03 23:51:45.973554: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpyozvude0 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/154f/1bc7/tmpu2nhzeiv.tmp" in 0.0017 seconds 2024-04-03 23:51:45.979074: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plantsgenomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmpurrgc9c /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmp1sznlek9.tmp" in 0.0015 seconds 2024-04-03 23:51:45.987535: Running the command: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmp1ad5htsj /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmpf10rqzda.tmp" 2024-04-03 23:51:45.993220: Successfully ran: "faffy chunk -c 30000000 -o 10000 --dir /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmp1ad5htsj /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/df31/d418/tmpf10rqzda.tmp" in 0.0013 seconds Issued job 'combine_chunks' kind-combine_chunks/instance-g67a6o8h v1 with job batch system ID: 595 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'combine_chunks' kind-combine_chunks/instance-5kkedfvy v1 with job batch system ID: 596 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Got message from job at time 04-03-2024 23:51:46: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_chunked_alignments/instance-jdzbjdv4/cleanup/file-3c656be7ce2d4d428a5c9adb6cb41761/stream used 0.00% disk (288.0 KiB [294912B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:46: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_chunked_alignments/instance-f20jgita/cleanup/file-692ec491e7874373b172225759b391f7/stream used 0.00% disk (288.0 KiB [294912B] used, 0 bB [0B] requested). Issued job 'make_ingroup_to_outgroup_alignments_3' kind-make_ingroup_to_outgroup_alignments_3/instance-gs2ktbya v1 with job batch system ID: 597 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'make_ingroup_to_outgroup_alignments_3' kind-make_ingroup_to_outgroup_alignments_3/instance-p0mgiiqh v1 with job batch system ID: 598 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:48.279227: Running the command: "paffy dechunk -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpaepspy0s.tmp --query --logLevel INFO" 2024-04-03 23:51:48.284060: Running the command: "paffy dechunk -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp1q4_55b7.tmp --query --logLevel INFO" 2024-04-03 23:51:48.284811: Successfully ran: "paffy dechunk -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpaepspy0s.tmp --query --logLevel INFO" in 0.0008 seconds with job-memory 2.0 Gi 2024-04-03 23:51:48.288097: Running the command: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpsg4f5bo5.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpk2uatwag.tmp" 2024-04-03 23:51:48.290067: Successfully ran: "paffy dechunk -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp1q4_55b7.tmp --query --logLevel INFO" in 0.0007 seconds with job-memory 2.0 Gi 2024-04-03 23:51:48.293342: Successfully ran: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpsg4f5bo5.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/2046/a732/tmpk2uatwag.tmp" in 0.0009 seconds with job-memory 2.0 Gi 2024-04-03 23:51:48.293405: Running the command: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp7fagwdpb.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp584qn5ew.tmp" 2024-04-03 23:51:48.299816: Successfully ran: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp7fagwdpb.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/199b/0207/tmp584qn5ew.tmp" in 0.0013 seconds with job-memory 2.0 Gi Got message from job at time 04-03-2024 23:51:48: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_ingroup_to_outgroup_alignments_3/instance-fotjq23s/cleanup/file-712152ac6d774382a3cac4e55c73d80c/stream used 0.00% disk (256.0 KiB [262144B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:48: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-make_ingroup_to_outgroup_alignments_3/instance-xhyqbkrf/cleanup/file-7d8370b615ee42cc89a56b08f30fae15/stream used 0.00% disk (256.0 KiB [262144B] used, 0 bB [0B] requested). Issued job 'invert_alignments' kind-invert_alignments/instance-12zrlv15 v1 with job batch system ID: 599 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:51.409165: Running the command: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c43e/7f1a/tmpe7m1w3ic.tmp" 2024-04-03 23:51:51.415462: Successfully ran: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c43e/7f1a/tmpe7m1w3ic.tmp" in 0.001 seconds with job-memory 2.0 Gi Issued job 'invert_alignments' kind-invert_alignments/instance-9zwixtb2 v1 with job batch system ID: 600 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:52.469376: Running the command: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/6b81/0bce/tmp2j0txejl.tmp" 2024-04-03 23:51:52.475852: Successfully ran: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/6b81/0bce/tmp2j0txejl.tmp" in 0.001 seconds with job-memory 2.0 Gi Issued job 'chain_alignments_splitting_ingroups_and_outgroups' kind-chain_alignments_splitting_ingroups_and_outgroups/instance-lfdokzx4 v1 with job batch system ID: 601 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'chain_alignments' kind-chain_alignments/instance-dggqyp0p v1 with job batch system ID: 602 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Issued job 'chain_alignments' kind-chain_alignments/instance-qojcwn7w v1 with job batch system ID: 603 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:56.864325: Running the command: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/5064/tmpa8mpb4d5/Anc21-Anc24_vs_Anc25.inv.paf" 2024-04-03 23:51:56.868263: Successfully ran: "paffy invert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/5064/tmpa8mpb4d5/Anc21-Anc24_vs_Anc25.inv.paf" in 0.0004 seconds 2024-04-03 23:51:56.870552: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/5064/tmpa8mpb4d5/Anc21-Anc24_vs_Anc25.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:56.876364: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/5064/tmpa8mpb4d5/Anc21-Anc24_vs_Anc25.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0004 seconds 2024-04-03 23:51:56.955142: Running the command: "paffy tile -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/chained_Anc21 --logLevel INFO" Issued job 'chain_one_alignment' kind-chain_one_alignment/instance-u4n2lecr v1 with job batch system ID: 604 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:56.961059: Successfully ran: "paffy tile -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/chained_Anc21 --logLevel INFO" in 0.0004 seconds 2024-04-03 23:51:56.963756: Running the command: "paffy trim -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/tiled_Anc21.paf --trimIdentity 0.2" Issued job 'chain_one_alignment' kind-chain_one_alignment/instance-137727r1 v1 with job batch system ID: 605 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:51:56.970163: Successfully ran: "paffy trim -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/tiled_Anc21.paf --trimIdentity 0.2" in 0.0003 seconds 2024-04-03 23:51:56.973008: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/trim_Anc21.paf --maxTileLevel 1" 2024-04-03 23:51:56.978876: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/trim_Anc21.paf --maxTileLevel 1" in 0.0004 seconds 2024-04-03 23:51:56.982078: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/filter_Anc21.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:56.987004: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/filter_Anc21.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0005 seconds 2024-04-03 23:51:56.989673: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/primary_chain_Anc21.paf --minChainScore 10000" 2024-04-03 23:51:56.994873: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c25/7a5a/tmpowt8wpv2/primary_chain_Anc21.paf --minChainScore 10000" in 0.0004 seconds Got message from job at time 04-03-2024 23:51:57: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-chain_one_alignment/instance-xipd0zsg/cleanup/file-3bb4932d0df44f1eba6023213f37bd06/stream used 0.00% disk (224.0 KiB [229376B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:57: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-tile_alignments/instance-umgs6o7k/cleanup/file-06865a9e228d4abc9a142d5f2896848c/stream used 0.00% disk (224.0 KiB [229376B] used, 0 bB [0B] requested). 2024-04-03 23:51:57.893844: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/3c16/f311/tmpje5xpcp0/Anc21-og_1.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:57.900801: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/3c16/f311/tmpje5xpcp0/Anc21-og_1.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0009 seconds 2024-04-03 23:51:57.903567: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/172e/819d/tmpi_i1kfsl/Anc21-og_0.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:57.909867: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/172e/819d/tmpi_i1kfsl/Anc21-og_0.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0008 seconds Issued job 'tile_alignments' kind-tile_alignments/instance-76jwbjyq v1 with job batch system ID: 606 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False Got message from job at time 04-03-2024 23:51:58: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-chain_one_alignment/instance-u4n2lecr/cleanup/file-887ac2c4ad704bf4a8b9153404692c6e/stream used 0.00% disk (176.0 KiB [180224B] used, 0 bB [0B] requested). Got message from job at time 04-03-2024 23:51:58: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-chain_one_alignment/instance-137727r1/cleanup/file-705e5bd7125f41c58c8e71d8e5cef71f/stream used 0.00% disk (176.0 KiB [180224B] used, 0 bB [0B] requested). 2024-04-03 23:51:59.024035: Running the command: "paffy tile -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/chained_Anc21 --logLevel INFO" 2024-04-03 23:51:59.029846: Successfully ran: "paffy tile -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/chained_Anc21 --logLevel INFO" in 0.0004 seconds 2024-04-03 23:51:59.033200: Running the command: "paffy trim -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/tiled_Anc21.paf --trimIdentity 0.2" 2024-04-03 23:51:59.037758: Successfully ran: "paffy trim -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/tiled_Anc21.paf --trimIdentity 0.2" in 0.0003 seconds 2024-04-03 23:51:59.040519: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/trim_Anc21.paf --maxTileLevel 1" 2024-04-03 23:51:59.045985: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/trim_Anc21.paf --maxTileLevel 1" in 0.0003 seconds 2024-04-03 23:51:59.049292: Running the command: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/filter_Anc21.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" 2024-04-03 23:51:59.054933: Successfully ran: "paffy chain -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/filter_Anc21.paf --maxGapLength 1000000 --chainGapOpen 5000 --chainGapExtend 1 --trimFraction 1.0 --logLevel INFO" in 0.0004 seconds 2024-04-03 23:51:59.057869: Running the command: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/primary_chain_Anc21.paf --minChainScore 10000" 2024-04-03 23:51:59.063785: Successfully ran: "paffy filter -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/b8d3/ffa6/tmprv2vh6l3/primary_chain_Anc21.paf --minChainScore 10000" in 0.0003 seconds Got message from job at time 04-03-2024 23:51:59: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-tile_alignments/instance-76jwbjyq/cleanup/file-b351972ac3504998b551c1c717ce9fd1/stream used 0.00% disk (224.0 KiB [229376B] used, 0 bB [0B] requested). Issued job 'merge_alignments' kind-merge_alignments/instance-lyqq6i5z v1 with job batch system ID: 607 and disk: 0, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:52:02.179147: Running the command: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/174b/b867/tmpvv9ra6e1/alignment_1.paf /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/174b/b867/tmpvv9ra6e1/alignment_2.paf" 2024-04-03 23:52:02.185738: Successfully ran: "cat /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/174b/b867/tmpvv9ra6e1/alignment_1.paf /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/174b/b867/tmpvv9ra6e1/alignment_2.paf" in 0.0014 seconds with job-memory 2.0 Gi Got message from job at time 04-03-2024 23:52:02: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-merge_alignments/instance-lyqq6i5z/cleanup/file-8fb0995b920e43a4aed508777cb70be2/stream used 0.00% disk (272.0 KiB [278528B] used, 0 bB [0B] requested). Issued job 'Job' kind-Job/instance-5gji0iw6 v1 with job batch system ID: 608 and disk: 95.4 Mi, memory: 488.3 Mi, cores: 0.1, accelerators: [], preemptible: False Issued job 'trim_unaligned_sequences' kind-trim_unaligned_sequences/instance-w1nlzryh v1 with job batch system ID: 609 and disk: 0, memory: 0, cores: 1, accelerators: [], preemptible: False 2024-04-03 23:52:07.996998: Running the command: "paffy to_bed --binary --excludeUnaligned --includeInverted -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmph1d404wy.tmp --logLevel INFO" 2024-04-03 23:52:08.004066: Successfully ran: "paffy to_bed --binary --excludeUnaligned --includeInverted -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmph1d404wy.tmp --logLevel INFO" in 0.0011 seconds 2024-04-03 23:52:08.012126: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmxn02h7e.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpq7fbqgre.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" 2024-04-03 23:52:08.018743: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmxn02h7e.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpq7fbqgre.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" in 0.0016 seconds 2024-04-03 23:52:08.027450: Running the command: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmxn02h7e.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmetfv5k9.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" 2024-04-03 23:52:08.032736: Successfully ran: "faffy extract -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmxn02h7e.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpmetfv5k9.tmp --skipMissing --minSize 1 --flank 2000 --logLevel INFO" in 0.0007 seconds 2024-04-03 23:52:08.037152: Running the command: "paffy upconvert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmph1d404wy.tmp --logLevel INFO /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpcwnxneus.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpkf27e6y5.tmp" 2024-04-03 23:52:08.042161: Successfully ran: "paffy upconvert -i /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmph1d404wy.tmp --logLevel INFO /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpcwnxneus.tmp /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/1c9d/6ccd/tmpkf27e6y5.tmp" in 0.0004 seconds Issued job 'progressive_step_2' kind-progressive_step_2/instance-4ekaouj2 v1 with job batch system ID: 610 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False Got message from job at time 04-03-2024 23:52:08: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/cleanup/file-f33d6aa793e64ad1bb7a0bd5a421c530/stream used 0.00% disk (400.0 KiB [409600B] used, 0 bB [0B] requested). Estimating cactus_consolidated(Anc21) memory as 3.7 Gi from 4 sequences with total-sequence-size 0 b and paf-size 0 using configuration settings Issued job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v1 with job batch system ID: 611 and disk: 0, memory: 3.7 Gi, cores: 40, accelerators: [], preemptible: False 2024-04-03 23:52:10.421692: Running the command: "cactus_consolidated --sequences Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa --speciesTree (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/An <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref --outgroupEvents Anc22 Anc20 --referenceEvent Anc21 --threads 40" cactus_consolidated(Anc21): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml cactus_consolidated(Anc21): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h cactus_consolidated(Anc21): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h.fa cactus_consolidated(Anc21): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref cactus_consolidated(Anc21): Sequence files and events: Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa cactus_consolidated(Anc21): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf cactus_consolidated(Anc21): Secondary alignments file: (null) cactus_consolidated(Anc21): Constraint alignments file: (null) cactus_consolidated(Anc21): Species tree: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; cactus_consolidated(Anc21): Outgroup events: Anc22 Anc20 cactus_consolidated(Anc21): Reference event: Anc21 cactus_consolidated(Anc21): Loaded the parameters files, 0 seconds have elapsed cactus_consolidated(Anc21): Set up the cactus disk, 0 seconds have elapsed cactus_consolidated(Anc21): Constructed the first flower cactus_consolidated(Anc21): Going to build the event tree with newick string: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; cactus_consolidated(Anc21): Parsed the tree cactus_consolidated(Anc21): Constructed the basic event tree cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa to Anc24 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa to Anc25 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa to Anc20 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa to Anc22 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa cactus_consolidated(Anc21): Constructed the initial flower with 0 sequences and 8 events with string: ((((Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:1,Anc20:1)Anc17:9.22337e+18)ROOT:9.22337e+18; cactus_consolidated(Anc21): Established the first Flower in the hierarchy, 0 seconds have elapsed cactus_consolidated(Anc21): Converted alignment coordinates, 0 seconds have elapsed cactus_consolidated(Anc21): Stripped any unique IDs, 0 seconds have elapsed cactus_consolidated(Anc21): cactus_consolidated: cactus_consolidated.c:351: main: Assertion !flower_builtBlocks(flower)' failed. Got exit code 1 (indicating failure) from job _toil_worker cactus_cons file:/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/jsTest_v270 kind-cactus_cons/instance-6b2yx4ez. Job failed with exit value 1: 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v1 Exit reason: None The job seems to have left a log file, indicating failure: 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v2 Log from job "kind-cactus_cons/instance-6b2yx4ez" follows: =========> [2024-04-03T23:52:10+0800] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2024-04-03T23:52:10+0800] [MainThread] [I] [toil] Running Toil version 5.12.0-6d5a5b83b649cd8adf34a5cfe89e7690c95189d3 on host comput187. [2024-04-03T23:52:10+0800] [MainThread] [I] [toil.worker] Working on job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v1 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil.worker] Loaded body Job('cactus_cons' kind-cactus_cons/instance-6b2yx4ez v1) from description 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v1 [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.statsAndLogging] Deprecated toil method that will be moved/replaced in a future release." [2024-04-03T23:52:10+0800] [MainThread] [W] [root] Deprecated toil method. Please call "logging.getLevelName" directly. [2024-04-03T23:52:10+0800] [MainThread] [I] [cactus.shared.common] Running the command ['cactus_consolidated', '--sequences', 'Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa', '--speciesTree', '(((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17;', '--logLevel', 'INFO', '--alignments', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf', '--params', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml', '--outputFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h', '--outputHalFastaFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h.fa', '--outputReferenceFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref', '--outgroupEvents', 'Anc22 Anc20', '--referenceEvent', 'Anc21', '--threads', '40'] [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] 2024-04-03 23:52:10.421692: Running the command: "cactus_consolidated --sequences Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa --speciesTree (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/An <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref --outgroupEvents Anc22 Anc20 --referenceEvent Anc21 --threads 40" [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Sequence files and events: Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Secondary alignments file: (null) [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constraint alignments file: (null) [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Species tree: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Outgroup events: Anc22 Anc20 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Reference event: Anc21 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Loaded the parameters files, 0 seconds have elapsed [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Set up the cactus disk, 0 seconds have elapsed [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the first flower [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Going to build the event tree with newick string: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Parsed the tree [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the basic event tree [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa to Anc24 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa to Anc25 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa to Anc20 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa to Anc22 [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the initial flower with 0 sequences and 8 events with string: ((((Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:1,Anc20:1)Anc17:9.22337e+18)ROOT:9.22337e+18; [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Established the first Flower in the hierarchy, 0 seconds have elapsed [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Converted alignment coordinates, 0 seconds have elapsed [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Stripped any unique IDs, 0 seconds have elapsed [2024-04-03T23:52:10+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): cactus_consolidated: cactus_consolidated.c:351: main: Assertion!flower_builtBlocks(flower)' failed. [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files: [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-t4adhxyj/file-60242bf832c4457eb68f015444c1db3b/Anc24.ref' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa' [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-a15d1wm4/file-00deaa915ee14971a937a22b854f511c/Anc25.ref' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa' [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-fe48981f168f458cafc5d71528e1ee1b/tmpkf27e6y5.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa' [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-387885a3514345438fd3d59a69f5a106/tmpcwnxneus.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa' [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-110648eddeea4306b7c443243d65a8d1/tmpnryn5pxb.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.paf' [2024-04-03T23:52:10+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] LOG-TO-MASTER: Job used more disk than requested. For CWL, consider increasing the outdirMin requirement, otherwise, consider increasing the disk requirement. Job files/for-job/kind-cactus_cons/instance-6b2yx4ez/cleanup/file-d52b4d2bb7524c82b188e75d057c8d05/stream used 0.00% disk (512.0 KiB [524288B] used, 0 bB [0B] requested). Traceback (most recent call last): File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/worker.py", line 403, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2774, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2691, in _run return self.run(fileStore) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2919, in run rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/cactus/pipeline/cactus_workflow.py", line 145, in cactus_cons job_memory=job.memory)[1] # Get just the standard error output File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/cactus/shared/common.py", line 893, in cactus_call raise RuntimeError("{}Command {} signaled {}: {}".format(sigill_msg, call, signal.Signals(-process.returncode).name, out)) RuntimeError: cactus_consolidated(Anc21): Command ['cactus_consolidated', '--sequences', 'Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc22.fa', '--speciesTree', '(((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17;', '--logLevel', 'INFO', '--alignments', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21_primary.paf', '--params', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.config.xml', '--outputFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h', '--outputHalFastaFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.c2h.fa', '--outputReferenceFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/e7cf/9220/tmpqgz9li0v/Anc21.ref', '--outgroupEvents', 'Anc22 Anc20', '--referenceEvent', 'Anc21', '--threads', '40'] signaled SIGABRT: [2024-04-03T23:52:10+0800] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host comput187 <========= Due to failure we are reducing the remaining try count of job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v2 with ID kind-cactus_cons/instance-6b2yx4ez to 1 We have increased the disk of the failed job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v2 to the default of 2147483648 bytes Issued job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v3 with job batch system ID: 612 and disk: 2.0 Gi, memory: 3.7 Gi, cores: 40, accelerators: [], preemptible: False 2024-04-03 23:52:11.480231: Running the command: "cactus_consolidated --sequences Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa --speciesTree (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/An <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref --outgroupEvents Anc22 Anc20 --referenceEvent Anc21 --threads 40" cactus_consolidated(Anc21): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml cactus_consolidated(Anc21): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h cactus_consolidated(Anc21): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h.fa cactus_consolidated(Anc21): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref cactus_consolidated(Anc21): Sequence files and events: Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa cactus_consolidated(Anc21): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf cactus_consolidated(Anc21): Secondary alignments file: (null) cactus_consolidated(Anc21): Constraint alignments file: (null) cactus_consolidated(Anc21): Species tree: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; cactus_consolidated(Anc21): Outgroup events: Anc22 Anc20 cactus_consolidated(Anc21): Reference event: Anc21 cactus_consolidated(Anc21): Loaded the parameters files, 0 seconds have elapsed cactus_consolidated(Anc21): Set up the cactus disk, 0 seconds have elapsed cactus_consolidated(Anc21): Constructed the first flower cactus_consolidated(Anc21): Going to build the event tree with newick string: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; cactus_consolidated(Anc21): Parsed the tree cactus_consolidated(Anc21): Constructed the basic event tree cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa to Anc24 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa to Anc25 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa to Anc20 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa to Anc22 cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa cactus_consolidated(Anc21): Constructed the initial flower with 0 sequences and 8 events with string: ((((Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:1,Anc20:1)Anc17:9.22337e+18)ROOT:9.22337e+18; cactus_consolidated(Anc21): Established the first Flower in the hierarchy, 0 seconds have elapsed cactus_consolidated(Anc21): Converted alignment coordinates, 0 seconds have elapsed cactus_consolidated(Anc21): Stripped any unique IDs, 0 seconds have elapsed cactus_consolidated(Anc21): cactus_consolidated: cactus_consolidated.c:351: main: Assertion !flower_builtBlocks(flower)' failed. Got exit code 1 (indicating failure) from job _toil_worker cactus_cons file:/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/jsTest_v270 kind-cactus_cons/instance-6b2yx4ez. Job failed with exit value 1: 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v3 Exit reason: None The job seems to have left a log file, indicating failure: 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v5 Log from job "kind-cactus_cons/instance-6b2yx4ez" follows: =========> [2024-04-03T23:52:11+0800] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG--- [2024-04-03T23:52:11+0800] [MainThread] [I] [toil] Running Toil version 5.12.0-6d5a5b83b649cd8adf34a5cfe89e7690c95189d3 on host comput187. [2024-04-03T23:52:11+0800] [MainThread] [I] [toil.worker] Working on job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v4 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil.worker] Loaded body Job('cactus_cons' kind-cactus_cons/instance-6b2yx4ez v4) from description 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v4 [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.statsAndLogging] Deprecated toil method that will be moved/replaced in a future release." [2024-04-03T23:52:11+0800] [MainThread] [W] [root] Deprecated toil method. Please call "logging.getLevelName" directly. [2024-04-03T23:52:11+0800] [MainThread] [I] [cactus.shared.common] Running the command ['cactus_consolidated', '--sequences', 'Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa', '--speciesTree', '(((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17;', '--logLevel', 'INFO', '--alignments', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf', '--params', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml', '--outputFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h', '--outputHalFastaFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h.fa', '--outputReferenceFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref', '--outgroupEvents', 'Anc22 Anc20', '--referenceEvent', 'Anc21', '--threads', '40'] [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] 2024-04-03 23:52:11.480231: Running the command: "cactus_consolidated --sequences Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa --speciesTree (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; --logLevel INFO --alignments /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf --params /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml --outputFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h --outputHalFastaFile /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/An <...> le /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref --outgroupEvents Anc22 Anc20 --referenceEvent Anc21 --threads 40" [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Params file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output hal fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Output reference fasta file string : /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Sequence files and events: Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Alignments file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Secondary alignments file: (null) [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constraint alignments file: (null) [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Species tree: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Outgroup events: Anc22 Anc20 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Reference event: Anc21 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Loaded the parameters files, 0 seconds have elapsed [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Set up the cactus disk, 0 seconds have elapsed [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the first flower [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Going to build the event tree with newick string: (((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17; [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Parsed the tree [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the basic event tree [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa to Anc24 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa to Anc25 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa to Anc20 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Assigning sequence /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa to Anc22 [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Processing file: /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Constructed the initial flower with 0 sequences and 8 events with string: ((((Anc24:1,Anc25:1)Anc21:1,Anc22:1)Anc19:1,Anc20:1)Anc17:9.22337e+18)ROOT:9.22337e+18; [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Established the first Flower in the hierarchy, 0 seconds have elapsed [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Converted alignment coordinates, 0 seconds have elapsed [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): Stripped any unique IDs, 0 seconds have elapsed [2024-04-03T23:52:11+0800] [MainThread] [I] [toil-rt] cactus_consolidated(Anc21): cactus_consolidated: cactus_consolidated.c:351: main: Assertion!flower_builtBlocks(flower)' failed. [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files: [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-t4adhxyj/file-60242bf832c4457eb68f015444c1db3b/Anc24.ref' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa' [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-cactus_cons/instance-a15d1wm4/file-00deaa915ee14971a937a22b854f511c/Anc25.ref' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa' [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-fe48981f168f458cafc5d71528e1ee1b/tmpkf27e6y5.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa' [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-387885a3514345438fd3d59a69f5a106/tmpcwnxneus.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa' [2024-04-03T23:52:11+0800] [MainThread] [W] [toil.fileStores.abstractFileStore] Downloaded file 'files/for-job/kind-trim_unaligned_sequences/instance-w1nlzryh/file-110648eddeea4306b7c443243d65a8d1/tmpnryn5pxb.tmp' to path '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.paf' Traceback (most recent call last): File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/worker.py", line 403, in workerScript job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2774, in _runner returnValues = self._run(jobGraph=None, fileStore=fileStore) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2691, in _run return self.run(fileStore) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/toil/job.py", line 2919, in run rValue = userFunction(*((self,) + tuple(self._args)), **self._kwargs) File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/cactus/pipeline/cactus_workflow.py", line 145, in cactus_cons job_memory=job.memory)[1] # Get just the standard error output File "/panfs4/gpu/home/songht/tools/cactus/binary_distribution/cactus-bin-v2.7.0/venv-cactus-v2.7.0/lib/python3.7/site-packages/cactus/shared/common.py", line 893, in cactus_call raise RuntimeError("{}Command {} signaled {}: {}".format(sigill_msg, call, signal.Signals(-process.returncode).name, out)) RuntimeError: cactus_consolidated(Anc21): Command ['cactus_consolidated', '--sequences', 'Anc24 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc24.fa Anc25 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc25.fa Anc20 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc20.fa Anc22 /panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc22.fa', '--speciesTree', '(((Anc24:1.0,Anc25:1.0)Anc21:1.0,Anc22:1.0)Anc19:1.0,Anc20:1.0)Anc17;', '--logLevel', 'INFO', '--alignments', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21_primary.paf', '--params', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.config.xml', '--outputFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h', '--outputHalFastaFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.c2h.fa', '--outputReferenceFile', '/panfs4/gpu/home/songht/plants_genomes/Juglandaceae/cactus/tmpTest/ba3f4b0629b55cacb548ae09f01cf731/c172/55dc/tmpag5xskmz/Anc21.ref', '--outgroupEvents', 'Anc22 Anc20', '--referenceEvent', 'Anc21', '--threads', '40'] signaled SIGABRT: [2024-04-03T23:52:11+0800] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host comput187 <========= Due to failure we are reducing the remaining try count of job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v5 with ID kind-cactus_cons/instance-6b2yx4ez to 0 Job 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v6 is completely failed Finished toil run with 18 failed jobs. Failed jobs at end of the run: 'progressive_step' kind-progressive_step/instance-o6pwv244 v2 'progressive_step' kind-progressive_step/instance-t3w9olwe v2 'sanitize_fasta_headers' kind-progressive_workflow/instance-miayy8dg v4 'EncapsulatedJob' kind-EncapsulatedJob/instance-gjs3eyi9 v2 'progressive_step' kind-progressive_step/instance-ct13tv4w v3 'progressive_step' kind-progressive_step/instance-1t757117 v3 'progressive_step' kind-progressive_step/instance-svke30a5 v2 'progressive_step' kind-progressive_step/instance-lokl715o v2 'Job' kind-progressive_schedule/instance-8nnj8kc9 v5 'progressive_step' kind-progressive_step/instance-vgidb01g v2 'progressive_step' kind-progressive_step/instance-rgvutydl v2 'Job' kind-Job/instance-5gji0iw6 v2 'Job' kind-preprocess_all/instance-x2398ld2 v5 'cactus_cons_with_resources' kind-progressive_step_2/instance-4ekaouj2 v5 'progressive_step' kind-progressive_step/instance-ryifewd7 v2 'cactus_cons' kind-cactus_cons/instance-6b2yx4ez v6 'progressive_step' kind-progressive_step/instance-lk4bgpjm v3 'progressive_step' kind-progressive_step/instance-_s8svaly v2 Stopping real-time logging server. Joining real-time logging server thread.

ferninfm commented 2 months ago

Hi Glenn and Song98*, just to say that this issue should be closed. I was trying to solve the same error in v. 2.8.0. The issue was solved in release 2.8.1 2024.04.04

Thanks for your work!

Fernando