Encountered network problems according to the tutorial

[user-tq]

According to the dna-seq-varlociraptor tutorial, I ran

snake-make -- cores all

I got a lot of network errors, I tried to access one of the links. https://github.com/snakemake/snakemake-wrappers/raw/v1.10.0/bio/bwa/index/wrapper.Rmd I get

I'm not sure if it's my network problem, or maybe it's the problem of dna-seq-varlocator? Or is it the problem of github? I really don't know who to ask. In addition, whether there is a better localized operation method, it is not easy to maintain a high-quality network, and whether all the required files can be downloaded at one time

[user-tq]

I tried to do some localization work

mkdir  down_dna-seq-varlociraptor/
cd down_dna-seq-varlociraptor/
wegt  -c https://github.com/snakemake-workflows/dna-seq-varlociraptor/archive/refs/tags/v3.22.0.zip
unzip dna-seq-varlociraptor-3.22.0.zip

and modify the file path in workflow/Snakefile But it still can't solve the problem. then，i down wrappers and use --wrapper-prefix

wget -c https://github.com/snakemake/snakemake-wrappers/archive/refs/tags/v1.23.5.zip
unzip  snakemake-wrappers-1.23.5.zip
snakemake --cores all    --wrapper-prefix  /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/

return erro:

Failed to open source file /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/v1.2.0/bio/reference/ensembl-sequence/environment.yaml

grep -rE "v[1-9].*/bio"|awk '{gsub(/"/,"");print $0}'

dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.3.2/bio/fastqc
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.10.0/bio/samtools/idxstats
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.10.0/bio/samtools/stats
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.21.2/bio/multiqc
dna-seq-varlociraptor-3.22.0/workflow/rules/regions.smk:        v1.12.0/bio/mosdepth
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.2.0/bio/sra-tools/fasterq-dump
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.12.0/bio/cutadapt/pe
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.12.0/bio/cutadapt/se
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.19.0/bio/freebayes
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.10.0/bio/delly
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.19.1/bio/bcftools/filter
dna-seq-varlociraptor-3.22.0/workflow/rules/primers.smk:        v1.10.0/bio/samtools/view
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.10.0/bio/bcftools/view
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.10.0/bio/samtools/index
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.12.0/bio/tabix/index
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.10.0/bio/samtools/sort
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.2.0/bio/benchmark/chm-eval-kit
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.2.0/bio/benchmark/chm-eval
dna-seq-varlociraptor-3.22.0/workflow/rules/calling.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.22.0/bio/vep/annotate
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.22.0/bio/vep/annotate
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/bwa/mem
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/fgbio/annotatebamwithumis
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/picard/markduplicates
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/bwa/mem
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/samtools/merge
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/samtools/sort
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/gatk/baserecalibratorspark
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/gatk/applybqsr
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.2.0/bio/reference/ensembl-sequence
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.10.0/bio/samtools/faidx
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.12.0/bio/reference/ensembl-variation
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.5.0/bio/reference/ensembl-annotation
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.10.0/bio/bwa/index
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.22.0/bio/vep/cache
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.12.0/bio/vep/plugins
dna-seq-varlociraptor-3.22.0/workflow/rules/testcase.smk:        v1.16.0/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/filtering.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/filtering.smk:        v1.14.1/bio/bcftools/concat

I checked all the software versions used by rule. In fact, I was confused about the meaning of this version. Is it the wrapper version?.Why use multiple versions? Or the version of the software itself. In short, I will replace them all with v1.23.5,and construction directory

 sed -i 's/v[1-9].*\/bio/v1.23.5\/bio/g'  dna-seq-varlociraptor-3.22.0/workflow/rules/*smk

finaly and I run

snakemake --cores all     --wrapper-prefix  /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/

I think I have solved the network problem of accessing github through localization, but now, it seems that I can't build the environment and use bgzip in it, which may be the reason for the network......

[user-tq]

I tried to do some localization work

mkdir  down_dna-seq-varlociraptor/
cd down_dna-seq-varlociraptor/
wegt  -c https://github.com/snakemake-workflows/dna-seq-varlociraptor/archive/refs/tags/v3.22.0.zip
unzip dna-seq-varlociraptor-3.22.0.zip

and modify the file path in workflow/Snakefile But it still can't solve the problem. then，i down wrappers and use --wrapper-prefix

wget -c https://github.com/snakemake/snakemake-wrappers/archive/refs/tags/v1.23.5.zip
unzip  snakemake-wrappers-1.23.5.zip
snakemake --cores all    --wrapper-prefix  /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/

return erro:

Failed to open source file /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/v1.2.0/bio/reference/ensembl-sequence/environment.yaml

grep -rE "v[1-9].*/bio"|awk '{gsub(/"/,"");print $0}'

dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.3.2/bio/fastqc
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.10.0/bio/samtools/idxstats
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.10.0/bio/samtools/stats
dna-seq-varlociraptor-3.22.0/workflow/rules/qc.smk:        v1.21.2/bio/multiqc
dna-seq-varlociraptor-3.22.0/workflow/rules/regions.smk:        v1.12.0/bio/mosdepth
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.2.0/bio/sra-tools/fasterq-dump
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.12.0/bio/cutadapt/pe
dna-seq-varlociraptor-3.22.0/workflow/rules/trimming.smk:        v1.12.0/bio/cutadapt/se
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.19.0/bio/freebayes
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.10.0/bio/delly
dna-seq-varlociraptor-3.22.0/workflow/rules/candidate_calling.smk:        v1.19.1/bio/bcftools/filter
dna-seq-varlociraptor-3.22.0/workflow/rules/primers.smk:        v1.10.0/bio/samtools/view
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.10.0/bio/bcftools/view
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.10.0/bio/samtools/index
dna-seq-varlociraptor-3.22.0/workflow/rules/utils.smk:        v1.12.0/bio/tabix/index
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.10.0/bio/samtools/sort
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.2.0/bio/benchmark/chm-eval-kit
dna-seq-varlociraptor-3.22.0/workflow/rules/benchmarking.smk:        v1.2.0/bio/benchmark/chm-eval
dna-seq-varlociraptor-3.22.0/workflow/rules/calling.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.22.0/bio/vep/annotate
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.22.0/bio/vep/annotate
dna-seq-varlociraptor-3.22.0/workflow/rules/annotation.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/bwa/mem
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/fgbio/annotatebamwithumis
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/picard/markduplicates
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/bwa/mem
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/samtools/merge
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.10.0/bio/samtools/sort
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/gatk/baserecalibratorspark
dna-seq-varlociraptor-3.22.0/workflow/rules/mapping.smk:        v1.2.0/bio/gatk/applybqsr
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.2.0/bio/reference/ensembl-sequence
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.10.0/bio/samtools/faidx
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.12.0/bio/reference/ensembl-variation
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.5.0/bio/reference/ensembl-annotation
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.10.0/bio/bwa/index
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.22.0/bio/vep/cache
dna-seq-varlociraptor-3.22.0/workflow/rules/ref.smk:        v1.12.0/bio/vep/plugins
dna-seq-varlociraptor-3.22.0/workflow/rules/testcase.smk:        v1.16.0/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/filtering.smk:        v1.14.1/bio/bcftools/concat
dna-seq-varlociraptor-3.22.0/workflow/rules/filtering.smk:        v1.14.1/bio/bcftools/concat

I checked all the software versions used by rule. In fact, I was confused about the meaning of this version. Is it the wrapper version?.Why use multiple versions? Or the version of the software itself. In short, I will replace them all with v1.23.5,and construction directory

 sed -i 's/v[1-9].*\/bio/v1.23.5\/bio/g'  dna-seq-varlociraptor-3.22.0/workflow/rules/*smk

finaly and I run

snakemake --cores all     --wrapper-prefix  /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/

I think I have solved the network problem of accessing github through localization, but now, it seems that I can't build the environment and use bgzip in it, which may be the reason for the network......

I'm so stupid. I forgot to use conda to create a running environment,Finally, I ran successfully.

snakemake --cores all     --wrapper-prefix  /mnt/script/tanq/snakemake_varlociraptor_work/down_wrappers/snakemake-wrappers-1.23.5/  --use-conda

[github-actions[bot]]

This issue was marked as stale because it has been open for 6 months with no activity.

[fgvieira]

Closing since it seems it is fixed.