search

CCBR / RENEE

A comprehensive quality-control and quantification RNA-seq pipeline
https://CCBR.github.io/RENEE/
MIT License
4 stars 4 forks source link

singularity pull rate limit #52

Closed kelly-sovacool closed 11 months ago

kelly-sovacool commented 11 months ago

Error run into by Ayslan on biowulf the first time he tried to run RENEE:

Failed to pull singularity image from docker://nciccbr/ccbr_rseqc_4.0.0:v1.0:
FATAL:   While making image from oci registry: error fetching image to cache: failed to get checksum for docker://nciccbr/ccbr_rseqc_4.0.0:v1.0: reading manifest v1.0 in docker.io/nciccbr/ccbr_rseqc_4.0.0: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Implementing the shared SIF cache dir by default should help so users don't have to pull containers already pulled by others.

kelly-sovacool commented 11 months ago

Ultimately the problem was the user was calling renee run once per fastq file instead of passing all fastq files at once. I could see how users may hit the dockerhub rate limit though if trying to run renee for multiple projects simultaneously.

kelly-sovacool commented 11 months ago

If other users run into legitimate rate limit issues due to running renee simultaneously on multiple data sets (rather than a user incorrectly running once per sample), we may want to look into how users can authenticate their dockerhub account on biowulf to increase their rate limit and add instructions to our docs. Until then, this is likely not a true issue.