search

perslab / CELLECT

CELLECT (CELL-type Expression-specific integration for Complex Traits)
GNU General Public License v3.0
71 stars 19 forks source link

can't get results with the error saying "some output files are missing" #86

Closed HGT-UwU closed 2 years ago

HGT-UwU commented 2 years ago

Here is the outcome after I conduct "snakemake --use-conda -j -s cellect-ldsc.snakefile --configfile config.yml", and in /CELLECT/CELLECT-EXAMPLE/CELLECT-LDSC, there are only /logs and /precomputation, no results.

Loaded config file from --configfile argument Building DAG of jobs... Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'. Using shell: /usr/bin/bash Provided cores: 28 Rules claiming more threads will be scaled down. Job stats: job count min threads max threads

10 22 1 1 all 1 1 1 compute_LD_scores 22 1 1 compute_LD_scores_all_genes 22 1 1 find_overlaps 22 1 1 format_genes 1 1 1 make_all_genes_background 1 1 1 make_annot 22 1 1 make_annot_all_genes 22 1 1 make_cts_file 1 1 1 parse_results 1 1 1 prioritize_annotations 1 1 1 total 138 1 1

Select jobs to execute...

[Fri Nov 25 08:11:44 2022] rule format_genes: input: /home/software/CELLECT/data/shared/gene_coordinates.GRCh37.ensembl_v91.txt, /home/software/CELLECT/data/ldsc/GRCh37-chr-sizes.txt output: CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.1.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.2.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.3.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.4.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.5.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.6.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.7.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.8.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.9.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.10.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.11.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.12.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.13.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.14.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.15.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.16.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.17.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.18.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.19.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.20.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.21.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.22.bed log: CELLECT-EXAMPLE/CELLECT-LDSC/logs/log.format_genes_snake.txt jobid: 8 reason: Missing output files: CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.19.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.22.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.13.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.2.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.16.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.5.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.9.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.12.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.17.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.7.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.15.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.14.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.6.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.4.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.20.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.11.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.8.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.3.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.1.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.10.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.21.bed, CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.18.bed wildcards: BASE_OUTPUT_DIR=CELLECT-EXAMPLE/CELLECT-LDSC resources: tmpdir=/tmp

[Fri Nov 25 08:11:44 2022] rule make_all_genes_background: input: /home/software/CELLECT/example/tabula_muris-test.csv output: CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/tabula_muris-test/all_genes.tabula_muris-test.csv jobid: 95 reason: Missing output files: CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/tabula_muris-test/all_genes.tabula_muris-test.csv wildcards: BASE_OUTPUT_DIR=CELLECT-EXAMPLE/CELLECT-LDSC, run_prefix=tabula_muris-test resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/6b0ed30928628c9f8e8d487cd5f68f9e Activating conda environment: .snakemake/conda/6b0ed30928628c9f8e8d487cd5f68f9e Environment defines Python version < 3.7. Using Python of the main process to execute script. Note that this cannot be avoided, because the script uses data structures from Snakemake which are Python >=3.7 only. Activating conda environment: .snakemake/conda/6b0ed30928628c9f8e8d487cd5f68f9e Environment defines Python version < 3.7. Using Python of the main process to execute script. Note that this cannot be avoided, because the script uses data structures from Snakemake which are Python >=3.7 only. Activating conda environment: .snakemake/conda/6b0ed30928628c9f8e8d487cd5f68f9e Waiting at most 5 seconds for missing files. MissingOutputException in rule format_genes in line 200 of /home/software/CELLECT/cellect-ldsc.snakefile: Job Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait: CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.1.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.2.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.3.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.4.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.5.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.6.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.7.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.8.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.9.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.10.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.11.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.12.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.13.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.14.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.15.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.16.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.17.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.18.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.19.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.20.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.21.bed CELLECT-EXAMPLE/CELLECT-LDSC/precomputation/bed/genes_plus_100kb.22.bed completed successfully, but some output files are missing. 8 [Fri Nov 25 08:11:51 2022] Finished job 95. 1 of 138 steps (1%) done Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2022-11-25T081140.411046.snakemake.log

Tobi1kenobi commented 2 years ago

It's failing at the very first rule. I think this may be an issue with the conda environment from the various warnings and errors.

Try deleting the .snakemake directory and running: conda config --set channel_priority strict

Then trying again

HGT-UwU commented 2 years ago

It turns out that I didn't download all the lfs files.