search

haniffalab / webatlas-pipeline

A data pipeline built in Nextflow to process spatial and single-cell experiment data for visualisation in WebAtlas
MIT License
46 stars 10 forks source link

Not able to run the pipeline using singularity #132

Closed ashishjain1988 closed 3 months ago

ashishjain1988 commented 4 months ago

Hi,

I am trying to run the multimodal workflow using the singularity profile but it is giving me error realted to python packages not installed. I checked and found that the command is not able to download the singulairty containers/file locally. Can you please help me with it?

Command: ./nextflow run multimodal.nf -params-file p262_multimodal-template.yaml -profile singularity

"Error: ERROR ~ Error executing process > 'process_label (visium-heart-disease-label.zarr)'

Caused by: Process process_label (visium-heart-disease-label.zarr) terminated with an error exit status (1)

Command executed:

integrate_image.py --label_image_path visium-heart-disease-label.zarr --offset 1000000 --out_filename reindexed-visium-heart-disease-label.ome.zarr

Command exit status: 1

Command output: (empty)

Command error: Traceback (most recent call last): File "/work/webatlas-pipeline/bin/integrate_image.py", line 3, in import fire ModuleNotFoundError: No module named 'fire'"

Singularity version : apptainer version 1.3.1-1.el8 nextflow version: nextflow version 24.04.3.5916

@davehorsfall

ashishjain1988 commented 3 months ago

Hi @dannda,

While running the pipeline using the updated code, I am again getting error. It seems like the scanpy package is not installed correctly in the singularity image and it is trying to load it locally. Here is the error I got:

[ec/0cb7d3] Ful…les:route_file (h5ad, 230828_CZIgutage_seqPooled230801_reducedMeta.h5ad) [100%] 1 of 1, failed: 1 ✘[- ] Full_pipeline:Process_images:Generate_image - [- ] Full_pipeline:Process_images:image_to_zarr - [- ] Full_pipeline:Process_images:ome_zarr_metadata - [- ] Full_pipeline:Output_to_config:Build_config - Pulling Singularity image docker://haniffalab/webatlas-pipeline-build-config:0.5.1 [cache /lab-share/RC-Data-Science-e2/Public/Ashish/p262_scRNASeq_Thiagarajah/webatlas-pipeline/work/singularity/haniffalab-webatlas-pipeline-build-config-0.5.1.img]ERROR ~ Error executing process > 'Full_pipeline:Process_files:route_file (h5ad, 230828_CZIgutage_seqPooled230801_reducedMeta.h5ad)'

Caused by: Process Full_pipeline:Process_files:route_file (h5ad, 230828_CZIgutage_seqPooled230801_reducedMeta.h5ad) terminated with an error exit status (1)

Command executed:

router.py --file_type h5ad --path 230828_CZIgutage_seqPooled230801_reducedMeta.h5ad --stem p262-scRNAseq --args '{"compute_embeddings":false}'

Command exit status: 1

Command output: (empty)

Command error: Traceback (most recent call last): File "/lab-share/RC-Data-Science-e2/Public/Ashish/p262_scRNASeq_Thiagarajah/webatlas-pipeline/bin/router.py", line 12, in from process_h5ad import h5ad_to_zarr File "/lab-share/RC-Data-Science-e2/Public/Ashish/p262_scRNASeq_Thiagarajah/webatlas-pipeline/bin/process_h5ad.py", line 12, in import scanpy as sc File "/usr/local/lib/python3.10/site-packages/scanpy/init.py", line 14, in from . import tools as tl File "/usr/local/lib/python3.10/site-packages/scanpy/tools/init.py", line 1, in from ..preprocessing import pca File "/usr/local/lib/python3.10/site-packages/scanpy/preprocessing/init.py", line 1, in from ._recipes import recipe_zheng17, recipe_weinreb17, recipe_seurat File "/usr/local/lib/python3.10/site-packages/scanpy/preprocessing/_recipes.py", line 8, in from ._deprecated.highly_variable_genes import ( File "/usr/local/lib/python3.10/site-packages/scanpy/preprocessing/_deprecated/highly_variable_genes.py", line 11, in from .._utils import _get_mean_var File "/usr/local/lib/python3.10/site-packages/scanpy/preprocessing/_utils.py", line 46, in def sparse_mean_var_minor_axis(data, indices, major_len, minor_len, dtype): File "/usr/local/lib/python3.10/site-packages/numba/core/decorators.py", line 234, in wrapper disp.enable_caching() File "/usr/local/lib/python3.10/site-packages/numba/core/dispatcher.py", line 863, in enable_caching self._cache = FunctionCache(self.py_func) File "/usr/local/lib/python3.10/site-packages/numba/core/caching.py", line 601, in init self._impl = self._impl_class(py_func) File "/usr/local/lib/python3.10/site-packages/numba/core/caching.py", line 337, in init raise RuntimeError("cannot cache function %r: no locator available " RuntimeError: cannot cache function 'sparse_mean_var_minor_axis': no locator available for file '/usr/local/lib/python3.10/site-packages/scanpy/preprocessing/_utils.py'

Work dir: /lab-share/RC-Data-Science-e2/Public/Ashish/p262_scRNASeq_Thiagarajah/webatlas-pipeline/work/ec/0cb7d34d45d8a79589ac7c4ffcdb88

Regards, Ashish Jain