Closed ksenia-krasheninnikova closed 5 months ago
nf-core lint
overall result: Passed :white_check_mark: :warning:Posted for pipeline commit d99f5cc
+| ✅ 130 tests passed |+
#| ❔ 19 tests were ignored |#
!| ❗ 3 tests had warnings |!
@gq1 The purpose of this PR is to try setting the borrowed NCBI API key to check if it solves request errors on tower.
I've done some digging. The environment variable NCBI_API_KEY
is only recognised by the Entrez CLI, not BioPython.
See this part of the stack-trace
File "/opt/MitoHiFi/findMitoReference.py", line 39, in find_full_mito
handle = Entrez.esearch(db="nucleotide",term=term, idtype="acc")
The documentation https://biopython.org/docs/1.75/api/Bio.Entrez.html#module-Bio.Entrez says there is an api_key
parameter that needs to be set.
MitoHifi itself (specifically the findMitoReference.py
script) needs to be changed to take the API key (from the command-line or the environment) and pass it to the Entrez
module:
api_key=...
to every Entrez function call like the one aboveEntrez.api_key = ...
at the beginning of the script
@muffato thank you very much for looking into it I misread the docs and hoped setting the env variable will be enough. In this case I can update the findMitoReference.py script but this will require rebuilding of the docker/singularity image which might take time for back and forth with the MitoHiFi maintainer. I suggest I switch off organelles_on flag in github testing for now and we release the pipeline as is with the further updates on organelles subworkflow in release 1.1 that would also contain the oatk module so it all adds up nicely.
@ksenia-krasheninnikova
Why you increased the runner from 8c to 16c? The 8c runner failed not because the space, the error message is:
curl: (6) Could not resolve host: darwin.cog.sanger.ac.uk
@gq1 Yes, I assumed it's some server configuration related issue which made it unapplicable. But also, the host that we used to run it when we made the release was the 16c one. I don't recall all the details but it must be because the smaller ones were not enough.
Here are the resources, can we give 4c or 8c a try again?
4 | 16 GB | 150 GB | Ubuntu 8 | 32 GB | 300 GB | Ubuntu, Windows 16 | 64 GB | 600 GB | Ubuntu, Windows
Worked from the second attempt on 8c. It didn't work on 4c initially throwing not enough disk space error.
I can't see 4c being used in the recent commit, 150G disk space, quite lots. Maybe we can give another try.
Works fine with 4c. I thought ubuntu-latest is 4c herein the confusion.
Pipeline logic slightly changed:
Thank you @gq1 and @muffato ! Merging into dev.
PR checklist
nf-core lint
).nextflow run . -profile test,docker --outdir <OUTDIR>
).docs/usage.md
is updated.docs/output.md
is updated.CHANGELOG.md
is updated.README.md
is updated (including new tool citations and authors/contributors).