search

markziemann / dee2

Digital Expression Explorer 2 (DEE2): a repository of uniformly processed RNA-seq data
http://dee2.io
GNU General Public License v3.0
39 stars 7 forks source link

Errors when running multiple parallel jobs in singularity image #21

Closed markziemann closed 6 years ago

markziemann commented 6 years ago

Getting some errors when running multiple singularity instances in the same container. Have a feeling it could be the way the tmp.html file is created which collides between instances. A random tmpfile number would fix it.

Session Stop (Error: Server aborted session: No such file or directory) ERR1204262.qc failed ascp download ERR1204262.qc Validate the SRA file ERR1204262.qc SRAfilesize ERR1204262.qc.sra md5sums do not match. Deleting and exiting 54 ERR1216070.qc

adding: ERR1099520/ERR1099520.log (deflated 76%) adding: ERR1099520/ERR1099520.qc (deflated 50%) adding: ERR1099520/ERR1099520.se.tsv (deflated 81%) adding: ERR1099520/volunteer_pipeline.sh (deflated 74%) sftp> put ERR1099520.drerio.zip Uploading ERR1099520.drerio.zip to /incoming/ERR1099520.drerio.zip 13 using user data Error, not enough arguments specified. Quitting 14

markziemann commented 6 years ago

So one error is being called because .qc files are sneaking into the queue. Will need to look at dee_pipeline.R

markziemann commented 6 years ago

Would be a good idea to add an identifier to tmp.html so that there aren't any name clashes

markziemann commented 6 years ago

Now using the TMPHTML var which appends a $RANDOM to prevent tmp file collision. Fixed in dev branch