search

UMMS-Biocore / dolphinnext

A graphical user interface for distributed data processing of high throughput genomics
https://dolphinnext.umassmed.edu
4 stars 1 forks source link

Permission issue in tutorial #322

Closed little-peter-pan closed 3 years ago

little-peter-pan commented 3 years ago

Hi, I have tried to run tutorial several times, but when it comes to the final running pipeline, it always shows permission denied like this: ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "mkdir -p /export/tests/test1/run3 && echo 'INFO: Run directory created.'" 2>&1 mkdir: cannot create directory ‘/export/tests’: Permission denied ERROR: Run directory cannot be created.\nLOG: mkdir: cannot create directory ‘/export/tests’: Permission denied I'm using WSL2 with Ubuntu and Docker in Windows 10. I have tried running docker with sudo, with root, and chmod the export folder, but none of them worked... Any idea about how to fix this problem?

onuryukselen commented 3 years ago

Hello, could you please enter docker terminal and run following commands and send me the output: chmod -R 777 /export ls -lha /

Also may I learn your docker command to initialize your docker.

little-peter-pan commented 3 years ago

Here is the output: total 84K drwxr-xr-x 1 root root 4.0K Feb 10 14:53 . drwxr-xr-x 1 root root 4.0K Feb 10 14:53 .. -rwxr-xr-x 1 root root 0 Feb 10 14:53 .dockerenv drwxrwxrwx 1 root root 4.0K Nov 3 18:12 .nextflow drwxr-xr-x 1 root root 4.0K Nov 3 18:07 bin drwxr-xr-x 2 root root 4.0K Apr 12 2016 boot drwxrwxrwx 1 root root 4.0K Nov 3 18:59 data drwxr-xr-x 9 root root 2.8K Feb 10 14:53 dev drwxr-xr-x 1 root root 4.0K Feb 10 14:53 etc drwxrwxrwx 6 opencpu 1000 4.0K Feb 10 14:17 export drwxr-xr-x 1 root root 4.0K Nov 3 18:49 home drwxr-xr-x 1 root root 4.0K Nov 3 18:09 lib drwxr-xr-x 2 root root 4.0K Oct 14 11:01 lib64 drwxr-xr-x 2 root root 4.0K Oct 14 11:00 media drwxr-xr-x 2 root root 4.0K Oct 14 11:00 mnt drwxr-xr-x 2 root root 4.0K Oct 14 11:00 opt dr-xr-xr-x 185 root root 0 Feb 10 14:53 proc drwx------ 1 root root 4.0K Nov 3 18:58 root drwxr-xr-x 1 root root 4.0K Nov 3 18:49 run drwxr-xr-x 1 root root 4.0K Nov 3 18:07 sbin drwxr-xr-x 2 root root 4.0K Oct 14 11:00 srv dr-xr-xr-x 11 root root 0 Feb 10 14:53 sys drwxrwxrwt 1 root root 4.0K Nov 3 18:58 tmp drwxr-xr-x 1 root root 4.0K Oct 14 11:00 usr drwxr-xr-x 1 root root 4.0K Nov 3 18:06 var

The docker command I used is: docker run --rm --name test --privileged -m 10G -p 8080:80 -v ~/export:/export -ti ummsbiocore/dolphinnext-studio /bin/bash

onuryukselen commented 3 years ago

Could you please change the ownership of /export folder inside docker. chown root: root /export Also you might try changing permission of ~/export in windows.

little-peter-pan commented 3 years ago

I have changed /export ownership to root and tried again, but still the same error message: ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "mkdir -p /export/run3 && echo 'INFO: Run directory created.'" 2>&1 mkdir: cannot create directory ‘/export/run3’: Permission denied ERROR: Run directory cannot be created.\nLOG: mkdir: cannot create directory ‘/export/run3’: Permission denied

Here is what is inside /export: total 20K drwxr-xr-x 5 root root 4.0K Feb 10 15:15 . drwxr-xr-x 1 root root 4.0K Feb 10 15:20 .. drwxr-xr-x 3 www-data www-data 4.0K Feb 10 15:15 .dolphinnext drwxr-xr-x 11 www-data www-data 4.0K Nov 3 18:12 dolphinnext drwxr-xr-x 7 mysql mysql 4.0K Feb 10 15:21 mysql

And it is exactly the same in my Ubuntu /export folder. I tried to run "mkdir -p /export/run3" directly in docker and the folder can be created, but further analysis is still not working. Not sure if this is due to ownership or something else...

nephantes commented 3 years ago

Looks like you are using docker@localhost user. You need to give the write permission to that user. Why don't you change the ownership of /export folder to docker? Or create a folder /export/test and change the ownership to "docker" user for that folder and use it

little-peter-pan commented 3 years ago

Yes, I just figured out to change to docker user, and it worked, thanks.

But here comes the new problem:

  1. When I tried to add fastq files to input_reads, it shows the error message like "Query error, please check your connection...", and I have to restart docker run and then was able to see those files (page refresh not helpful). Not sure what caused this, but at least there is a workaround.

  2. Then the run failed in both docker image and singularity image: --- The docker version log output: ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "mkdir -p /export/run1 && echo 'INFO: Run directory created.'" 2>&1 INFO: Run directory created. rsync -e 'ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky' ../tmp/pub/ssRieZBHw8e9MPFdbuEehwLmdUCXTU/run.tar.gz docker@localhost:/export/run1 2>&1 ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost " test -f '/export/run1/run.tar.gz' && echo 'INFO: Run package exists.'" 2>&1 INFO: Run package exists. ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "source /etc/profile && tar xf /export/run1/run.tar.gz -C /export/run1 && rm /export/run1/run.tar.gz && bash /export/run1/.dolphinnext.init" >> ../tmp/pub/ssRieZBHw8e9MPFdbuEehwLmdUCXTU/run/serverlog.txt 2>&1 & echo $! & cp /export/run1/log.txt /export/run1/log.txt.1 2>/dev/null || true && >/export/run1/log.txt && mv /export/run1/timeline.html /export/run1/timeline.html.1 2>/dev/null || true && mv /export/run1/trace.txt /export/run1/trace.txt.1 2>/dev/null || true && mv /export/run1/dag.html /export/run1/dag.html.1 2>/dev/null || true && mv /export/run1/report.html /export/run1/report.html.1 2>/dev/null || true && mv /export/run1/.nextflow.log /export/run1/.nextflow.log.1 2>/dev/null || true && mv /export/run1/err.log /export/run1/err.log.1 2>/dev/null || true && mv /export/run1/out.log /export/run1/out.log.1 2>/dev/null || true && cp /export/run1/initialrun/initial.log /export/run1/initialrun/initial.log.1 2>/dev/null || true && >/export/run1/initialrun/initial.log && mv /export/run1/initialrun/timeline.html /export/run1/initialrun/timeline.html.1 2>/dev/null || true && mv /export/run1/initialrun/trace.txt /export/run1/initialrun/trace.txt.1 2>/dev/null || true && mv /export/run1/initialrun/dag.html /export/run1/initialrun/dag.html.1 2>/dev/null || true && mv /export/run1/initialrun/report.html /export/run1/initialrun/report.html.1 2>/dev/null || true && mv /export/run1/initialrun/.nextflow.log /export/run1/initialrun/.nextflow.log.1 2>/dev/null || true && mv /export/run1/initialrun/err.log /export/run1/initialrun/err.log.1 2>/dev/null || true && mv /export/run1/initialrun/out.log /export/run1/initialrun/out.log.1 2>/dev/null || true && mkdir -p /export/run1/.dolphinnext/uuid && touch /export/run1/.dolphinnext/uuid/ssRieZBHw8e9MPFdbuEehwLmdUCXTU && export NXF_VER=19.10.0 && export NXF_ANSI_LOG=false && source /etc/profile && NXF_SINGULARITY_CACHEDIR="${NXF_SINGULARITY_CACHEDIR:-$HOME/.dolphinnext/singularity}" && export NXF_SINGULARITY_CACHEDIR=$NXF_SINGULARITY_CACHEDIR && rm -rf /export/report1 && cd /export/run1/initialrun && nextflow /export/run1/initialrun/nextflow.nf -with-report -with-trace -with-timeline > /export/run1/initialrun/initial.log && cd /export/run1 && nextflow /export/run1/nextflow.nf -with-report -with-trace -with-timeline > /export/run1/log.txt && rm -rf /export/run1/work N E X T F L O W ~ version 19.10.0 Launching /export/run1/initialrun/nextflow.nf [loving_dubinsky] - revision: 7c01fc7e5b [4b/fb4ebd] Submitted process > initialCheck (1) [7e/4beb8c] Submitted process > createCollection (3) [0b/4c37be] Submitted process > createCollection (1) [53/83a105] Submitted process > createCollection (5) [b1/4e6135] Submitted process > createCollection (2) [0d/9c7af1] Submitted process > createCollection (4) [09/bfa6a7] Submitted process > createCollection (6) [99/96e553] Submitted process > cleanUp (1)

    Initial run summary

    Completed at: 2021-02-10T21:41:51.717Z

    Duration: 3.8s

    Success: PASSED

    Exit status: 0

    Waiting for the Next Run..

    N E X T F L O W ~ version 19.10.0 Launching /export/run1/nextflow.nf [thirsty_ptolemy] - revision: 31a51a5ae2 [11/1df8d3] Submitted process > FastQC (6) [d1/4b276b] Submitted process > FastQC (5) Error executing process > 'FastQC (6)'

Caused by: Process FastQC (6) terminated with an error exit status (127)

Command executed:

fastqc control_rep3.fastq

Command exit status: 127

Command output: (empty)

Command error: .command.run: line 240: docker: command not found

Work dir: /export/run1/work/11/1df8d33f3bafdef99ebea9a4de33a9

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line

Pipeline execution summary

Completed at: 2021-02-10T21:41:55.621Z

Duration: 1s

Success: failed

Exit status: 127

WARN: Killing pending tasks (1)

--- The singularity version log output is even longer, but something like this: ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "mkdir -p /export/run3 && echo 'INFO: Run directory created.'" 2>&1 INFO: Run directory created. rsync -e 'ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky' ../tmp/pub/WeJhHrRrJ8lSgDD4rTqoOmGgcNi2Z3/run.tar.gz docker@localhost:/export/run3 2>&1 ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost " test -f '/export/run3/run.tar.gz' && echo 'INFO: Run package exists.'" 2>&1 INFO: Run package exists. ssh -oStrictHostKeyChecking=no -q -oChallengeResponseAuthentication=no -oBatchMode=yes -oPasswordAuthentication=no -oConnectTimeout=3 -i /export/.dolphinnext/.ssh/1_1_ssh_pri.pky docker@localhost "source /etc/profile && tar xf /export/run3/run.tar.gz -C /export/run3 && rm /export/run3/run.tar.gz && bash /export/run3/.dolphinnext.init" >> ../tmp/pub/WeJhHrRrJ8lSgDD4rTqoOmGgcNi2Z3/run/serverlog.txt 2>&1 & echo $! & cp /export/run3/log.txt /export/run3/log.txt.2 2>/dev/null || true && >/export/run3/log.txt && mv /export/run3/timeline.html /export/run3/timeline.html.2 2>/dev/null || true && mv /export/run3/trace.txt /export/run3/trace.txt.2 2>/dev/null || true && mv /export/run3/dag.html /export/run3/dag.html.2 2>/dev/null || true && mv /export/run3/report.html /export/run3/report.html.2 2>/dev/null || true && mv /export/run3/.nextflow.log /export/run3/.nextflow.log.2 2>/dev/null || true && mv /export/run3/err.log /export/run3/err.log.2 2>/dev/null || true && mv /export/run3/out.log /export/run3/out.log.2 2>/dev/null || true && cp /export/run3/initialrun/initial.log /export/run3/initialrun/initial.log.2 2>/dev/null || true && >/export/run3/initialrun/initial.log && mv /export/run3/initialrun/timeline.html /export/run3/initialrun/timeline.html.2 2>/dev/null || true && mv /export/run3/initialrun/trace.txt /export/run3/initialrun/trace.txt.2 2>/dev/null || true && mv /export/run3/initialrun/dag.html /export/run3/initialrun/dag.html.2 2>/dev/null || true && mv /export/run3/initialrun/report.html /export/run3/initialrun/report.html.2 2>/dev/null || true && mv /export/run3/initialrun/.nextflow.log /export/run3/initialrun/.nextflow.log.2 2>/dev/null || true && mv /export/run3/initialrun/err.log /export/run3/initialrun/err.log.2 2>/dev/null || true && mv /export/run3/initialrun/out.log /export/run3/initialrun/out.log.2 2>/dev/null || true && mkdir -p /export/run3/.dolphinnext/uuid && touch /export/run3/.dolphinnext/uuid/WeJhHrRrJ8lSgDD4rTqoOmGgcNi2Z3 && export NXF_VER=19.10.0 && export NXF_ANSI_LOG=false && source /etc/profile && NXF_SINGULARITY_CACHEDIR="${NXF_SINGULARITY_CACHEDIR:-$HOME/.dolphinnext/singularity}" && export NXF_SINGULARITY_CACHEDIR=$NXF_SINGULARITY_CACHEDIR && mkdir -p $NXF_SINGULARITY_CACHEDIR && cd $NXF_SINGULARITY_CACHEDIR && if [ ! -f $NXF_SINGULARITY_CACHEDIR/dolphinnext-rnaseq-1.0.simg ]; then singularity pull --name dolphinnext-rnaseq-1.0.simg docker://dolphinnext/rnaseq:1.0; fi && mkdir -p $NXF_SINGULARITY_CACHEDIR && cd $NXF_SINGULARITY_CACHEDIR && if [ ! -f $NXF_SINGULARITY_CACHEDIR/UMMS-Biocore-initialrun-09.06.2020.simg ]; then wget https://galaxyweb.umassmed.edu/pub/dolphinnext_singularity/UMMS-Biocore-initialrun-09.06.2020.simg; fi && find $NXF_SINGULARITY_CACHEDIR -type f -regex '.UMMS-Biocore-initialrun.img' -not -name 'UMMS-Biocore-initialrun-09.06.2020.simg' -delete && rm -rf /export/report3 && cd /export/run3/initialrun && nextflow /export/run3/initialrun/nextflow.nf -with-report -with-trace -with-timeline > /export/run3/initialrun/initial.log && cd /export/run3 && nextflow /export/run3/nextflow.nf -with-report -with-trace -with-timeline > /export/run3/log.txt && rm -rf /export/run3/work --2021-02-10 21:22:36-- https://galaxyweb.umassmed.edu/pub/dolphinnext_singularity/UMMS-Biocore-initialrun-09.06.2020.simg Resolving galaxyweb.umassmed.edu (galaxyweb.umassmed.edu)... 205.172.168.17 Connecting to galaxyweb.umassmed.edu (galaxyweb.umassmed.edu)|205.172.168.17|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 614080543 (586M) [text/plain] Saving to: ‘UMMS-Biocore-initialrun-09.06.2020.simg’

 0K .......... .......... .......... .......... ..........  0%  555K 18m1s
 ......

599650K .......... .......... .......... ........ 100% 24.2M=88s

2021-02-10 21:24:05 (6.62 MB/s) - ‘UMMS-Biocore-initialrun-09.06.2020.simg’ saved [614080543/614080543] N E X T F L O W ~ version 19.10.0 Launching /export/run3/initialrun/nextflow.nf [stupefied_perlman] - revision: 7c01fc7e5b [ce/0d6c55] Submitted process > initialCheck (1) Error executing process > 'initialCheck (1)'

Caused by: Process initialCheck (1) terminated with an error exit status (255)

Command executed:

!/usr/bin/env perl

...... Command exit status: 255

Command output: (empty)

Command error: FATAL:  container creation failed: mount /proc/self/fd/6->/usr/local/var/singularity/mnt/session/rootfs error: can't mount image /proc/self/fd/6: failed to mount squashfs filesystem: invalid argument

Work dir: /export/run3/initialrun/work/ce/0d6c55aaf812ea36d5ff3b5e86d335

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

Initial run summary

Completed at: 2021-02-10T21:24:10.064Z

Duration: 2s

Success: failed

Exit status: 255

Waiting for the Next Run..

I just tried to run fastqc, and not doing other two, and it seems they can not run fastqc or even initial check...

onuryukselen commented 3 years ago

It looks like running singularity inside of the docker container triggered that error. Here, it was reported before: https://github.com/hpcng/singularity/issues/5408 They suggested to updating singularity version. We will update singularity version in our docker image, but for now you can follow these steps in your docker container to test this:

  1. Remove existing singularity: https://sylabs.io/guides/3.0/user-guide/installation.html#remove-an-old-version
  2. Remove old Go: rm -rvf /usr/local/go/
  3. Install latest go v1.3: export VERSION=1.13 OS=linux ARCH=amd64 && \ wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz && \ sudo tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz && \ rm go$VERSION.$OS-$ARCH.tar.gz && export PATH=$PATH:/usr/local/go/bin
  4. apt-get install -y cryptsetup
  5. Then install new singularity 3.6: https://sylabs.io/guides/3.6/user-guide/quick_start.html#download-singularity-from-a-release
little-peter-pan commented 3 years ago

Hi, following your instruction, the issue was successfully resolved. Thank you!