An error with docker image in Singularity

[shodais]

---

Thank you for establishing your technology, universc. Please help me. I use Mac OS and try to use universc with docker image in Singularity, but the following error is occurred.

Singularity> bash /universc/launch_universc.sh
Running launch_universc.sh in '/universc'
Error: Trying to run Cell Ranger installed at /cellranger-3.0.2.9001/cellranger
launch_universc.sh can only run with Cell Ranger installed locally
Install Cell Ranger in a directory with write permissions such as /home/shodai_s_16/local and export to the PATH
The following versions of Cell Ranger are found:
 cellranger: /cellranger-3.0.2.9001/cellranger /cellranger-3.0.2.9001/cellranger-cs/3.0.2.9001/bin/cellranger

Should I need to install Cell Ranger even though I use docker image? I could use universc in Docker but not in Singularity. Is this problem only with singularity?

[kbattenb]

Hi shodais,

Glad to see people showing interest in using UniverSC.

To answer your question, no. You do not need to install cellranger on your Mac directly so long as you have your Docker image.

You are getting this error message because the directory under cellranger that contains all the barcode files is not writable (at least it seems that way when executed through Singularity).

Sorry to say, we have not tried running UniverSC through Singluarity. To determine whether this is due to an issue with UniverSC or due to running UniverSC through Singularity, can you run UniverSC through Docker as it was originally designed?

Please let us know if you still run into issues after this.

Kai Battenberg

[TomKellyGenetics]

For diagnosing the issue, please run the following in your Singularity container:

which cellranger
echo $PATH

These dependencies should be pre-installed in the image on DockerHub but I've not tested it on Singularity.

Thank you for reporting this issue. I too am pleased to see interest in using our software.

[shodais]

Thank you for your replies.

To determine whether this is due to an issue with UniverSC or due to running UniverSC through Singularity, can you run UniverSC through Docker as it was originally designed?

Yes, I can run UniverSC through Docker desktop(local). However, It took too long time, so I try to use UniverSC through Singularity in my lab's remote server.

For diagnosing the issue, please run the following in your Singularity container:

Singularity> which cellranger
/cellranger-3.0.2.9001/cellranger
Singularity> echo $PATH
universc:/cellranger-3.0.2.9001:/cellranger-3.0.2.9001/cellranger-cs/3.0.2.9001/bin/:/cellranger-3.0.2.9001/cellranger-cs/3.0.2.9001/lib/bin:/cellranger-3.0.2.9001/cellranger-cs/3.0.2.9001/tenkit/bin/:/cellranger-3.0.2.9001/cellranger-cs/3.0.2.9001/tenkit/lib/bin:/martian/bin/:/.cargo/bin:/root/.cargo/bin/:/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Now, I am using UniverSC not through Singularity, but through command line in the remote server, in which Cell Ranger has been already installed. Thank you for trying to solve my problem.

[kbattenb]

It gives me a peace of mind to know that UniverSC is working through Docker. So, thank you for letting us know.

Sorry about Singularity.

Kai Battenberg

[TomKellyGenetics]

@shodais Suzuki-san, thank you for reporting this issue. I think I have finally found a solution.

While running UniverSC via Nextflow I had similar permissions issues and solved them here. https://github.com/nf-core/modules/pull/1706 https://github.com/nf-core/modules/pull/1706/commits/9a59cd004c9a214bb296a89392d239c206256cbe

Based on this solution I added this configuration to my workflow parameters. https://github.com/nextflow-io/nextflow/issues/1295#issuecomment-535000561

    container 'tomkellygenetics/universc:1.2.3'
    containerOptions = "--user root"

The root user within the container has write permissions and is able to change parameters for Cell Ranger as required. This is the default for docker and podman runs that we tested on. Fixing this avoid the issue running in docker containers via Nextflow and automated checks by nf-core GitHub Actions workflows.

Therefore I think this fix will work for Singularity as well. Apologies that it took almost a year to discover this workaround.

[TomKellyGenetics]

Given that UniverSC runs via Singularity on GitHub actions (https://github.com/nf-core/modules/actions/runs/3248319559/jobs/5329390232) and I resolved the same issue when running Nextflow with PROFILE=docker, I think the above solution should resolve this issue. I'll keep the issue open for other Singularity users to view it and close the issue after documentation is updated to address it.

[TomKellyGenetics]

I tested running UniverSC via Singularity without root privileges. Here are my options for Docker and Singularity to run in Nextflow.

    container "nfcore/universc:1.2.4"
    if (workflow.containerEngine == 'docker'){
        containerOptions = "--user root"
    }
    if (workflow.containerEngine == 'singularity'){
        containerOptions = "--writable"
    }

Use singularity --writeable to run UniverSC without errors. I've updated the "dev" branch with version 1.2.5-dev which modifies permissions and default users in the Docker image to avoid this problem in the future. See this issue on nextflow modules for more discussion: https://github.com/nf-core/modules/pull/1706#discussion_r1017569028

[Davidwei7]

Dear Sir/Madam,

Hope you are well.

Sorry for the duplicated messages and posts on the same issue, but maybe posting here would get more noticed.

I am using the universc software on our cluster. We have a rather complex system running on the cluster, even docker image is not supported on our cluster. But singularity is supported. So for convenience, I downloaded the docker image of universc and converted it to singularity image. But it has problem running (see copied image below).

I have also tried to build sandbox with --writable then it created a directory of the singularity image, which then I can navigate the cellranger and cp it to my home directory and then export path after putting it into the home directory, the go back to run it in my scratch place, it still does not work and give me errors like this:

do you think this is what I expect? any chance it will provide a singularity image for users?

It would be absolutely fantastic if you could give us some advice on how to handle this. We need to do an essential analysis on some published datasets, but they are from 4 different scRNA-Seq platforms, and your tool is probably our best hope to run such analysis, and would be super fantastic if we can use them and apply in our study, and we are planning to do some cool things in the wet-lab based on the result of this integrative analysis, and hope to answer some really key questions in human neurodevelopment.

Thank you in advance.

Best Wishes,

David

[TomKellyGenetics]

Hi David,

I’m pleased to hear you are interested to use our tool, it sounds like a use-case we had in mind while publishing it. Sorry I’m not sure how much I can help without understanding the specific settings of your IT system but I’ll try.

I’ve managed to run singularity and nextflow using our Docker image without modifications. If you aren’t able to do this it may be an issue with your system or the converted image. As discussed above the cellranger directory must be writeable so some singularity options need to be invoked to run it. The new UniverSC v1.2.5.1 image should address some of these issues as we modified it for Nextflow compatibility.

In case it helps to build a singularity image, I’ll share this information. UniverSC uses a custom base image maintained separately. You may need to build this as well.

https://hub.docker.com/r/tomkellygenetics/cellranger_clean https://github.com/TomKellyGenetics/cellranger_clean

We’ve named our tool distinct from Cell Ranger to clarify we are not affiliated with 10X who trademarked it. This is an open source implementation maintained separately. Unfortunately the Cell Ranger base image relies on Python 2 so it is difficult to update but it is archived here and still works. Building it again with the correct dependency versions could be troublesome I am afraid.

Thanks,

Tom K

[TomKellyGenetics]

By the way, another possible solution (if your system allows it) is to force the user settings to root within the container. This is the default for Docker images but Singularity supports this:

singularity run —user root …

[Davidwei7]

Hi Tom, Thank you so much. The image is now working on my cluster. Best Wishes, David