search

vib-singlecell-nf / vsn-pipelines

A repository of pipelines for single-cell data in Nextflow DSL2
GNU General Public License v3.0
75 stars 32 forks source link

docker error unknown shorthand flag: 'B' in -B while running the pipeline #377

Closed MarcDalod closed 2 years ago

MarcDalod commented 2 years ago

Hi All,

I am trying to run the nextflow multiruns pyscenic pipeline on a remote server for the small dataset given here (https://vsn-pipelines-examples.readthedocs.io/en/latest/PBMC10k_multiruns.html). As this remote server forces to use singularity image, therefore I built a dockerfile and then converted it to a singularity image from a docker image. The singularity image contain nextflow version 21.04.3, with java version 8 and docker version (17.03) was also installed within this docker image for making docker pull.

Now when I used this pipeline on the remote server using singularity I am having following problems,

  1. Pipeline Configuration:

I first pulled all the pipelines using the code given below:

nextflow pull vib-singlecell-nf/vsn-pipelines -r v0.25.0
  1. Then, I made the config file using the code given here (https://vsn-pipelines-examples.readthedocs.io/en/latest/PBMC10k_multiruns.html)
nextflow config vib-singlecell-nf/vsn-pipelines \
    -profile tenx,single_sample_scenic,scenic_multiruns,scenic_use_cistarget_motifs,scenic_use_cistarget_tracks,hg38,singularity \
    > pbmc10k.vsn-pipelines.complete.config
  1. Then I entered into the singularity container using, 
    
cheema@gauss07:~/scenic_analysis$ singularity shell -B /home/cheema:/home/cheema nf_doc_sing.img 

4. Then I run the nextflow command but It gave an error which has been shown below,

Singularity> nextflow -C pbmc10k.vsn-pipelines.complete.config \ run vib-singlecell-nf/vsn-pipelines \ -entry single_sample \ -r v0.25.0 -resume


4. Then I got the error given below,
 Complete error output is avaiable here (https://pastebin.com/xkuXH6qK)
I think the problem is caused due to the docker which is not able to recognize the -B option given to it in the config file.

Below is the config file used for this analysis:

manifest { name = 'vib-singlecell-nf/vsn-pipelines' description = 'A repository of pipelines for single-cell data in Nextflow DSL2' homePage = 'https://github.com/vib-singlecell-nf/vsn-pipelines' version = '0.25.0' mainScript = 'main.nf' defaultBranch = 'master' nextflowVersion = '!>=20.10.0' }

params { global { project_name = '10x_PBMC' outdir = 'out' species = 'human' genome { assembly = 'hg38' } } misc { test { enabled = false } } utils { container = 'vibsinglecellnf/utils:0.4.0' publish { compressionLevel = 6 annotateWithBatchVariableName = false mode = 'link' } } sc { file_converter { off = 'h5ad' tagCellWithSampleId = true remove10xGEMWell = false useFilteredMatrix = true makeVarIndexUnique = false } scanpy { container = 'vibsinglecellnf/scanpy:0.5.2' report { annotations_to_plot = [] } feature_selection { report_ipynb = '/src/scanpy/bin/reports/sc_select_variable_genes_report.ipynb' method = 'mean_disp_plot' minMean = 0.0125 maxMean = 3 minDisp = 0.5 off = 'h5ad' } feature_scaling { method = 'zscore_scale' maxSD = 10 off = 'h5ad' } neighborhood_graph {

         nPcs = 2
         off = 'h5ad'
     }
     dim_reduction {
        report_ipynb = '/src/scanpy/bin/reports/sc_dim_reduction_report.ipynb'
        pca {
           method = 'pca'
           nComps = 2
           off = 'h5ad'
        }
        umap {
           method = 'umap'
           off = 'h5ad'
        }
        tsne {
           method = 'tsne'
           off = 'h5ad'
        }
     }
     clustering {
        preflight_checks = true
        report_ipynb = '/src/scanpy/bin/reports/sc_clustering_report.ipynb'
        method = 'louvain'
        resolution = 0.8
        off = 'h5ad'
     }
     marker_genes {
        method = 'wilcoxon'
        ngenes = 0
        groupby = 'louvain'
        off = 'h5ad'
     }
     filter {
        report_ipynb = '/src/scanpy/bin/reports/sc_filter_qc_report.ipynb'
        cellFilterStrategy = 'fixedthresholds' 
        cellFilterMinNGenes = 1
        cellFilterMaxNGenes = 4000
        cellFilterMaxPercentMito = 0.15
        geneFilterMinNCells = 3
        off = 'h5ad'
        outdir = 'out'

     }
     data_transformation {
        method = 'log1p'
        off = 'h5ad'
     }
     normalization {
        method = 'cpx'
        countsPerCellAfter = 10000
        off = 'h5ad'
     }
  }
  scenic {
     container = 'aertslab/pyscenic:0.10.4'
     scenicoutdir = 'out/scenic/'
     filteredLoom = ''
     scenicOutputLoom = 'SCENIC_output.loom'
     scenicScopeOutputLoom = 'SCENIC_SCope_output.loom'
     mode = 'dask_multiprocessing'
     client_or_address = ''
     cell_id_attribute = 'CellID'
     gene_attribute = 'Gene'
     report_ipynb = '/src/scenic/bin/reports/scenic_report.ipynb'
     skipReports = false
     grn {
        algorithm = 'grnboost2'
        tfs = '/ddn1/vol1/staging/leuven/stg_00002/lcb/cflerin/resources/allTFs_hg38.txt'
     }
     cistarget {
        adj = 'adj.tsv'
        type = ''
        rank_threshold = 5000
        auc_threshold = 0.05
        nes_threshold = 3.0
        min_orthologous_identity = 0.0
        max_similarity_fdr = 0.001
        annotations_fname = ''
        thresholds = '0.75,0.90'
        top_n_targets = 50
        top_n_regulators = '5,10,50'
        min_genes = 20
        all_modules = false
        motifsDb = '/staging/leuven/res_00001/databases/cistarget/databases/homo_sapiens/hg38/refseq_r80/mc9nr/gene_based/hg38__refseq-r80__*feather'
        motifsAnnotation = '/staging/leuven/res_00001/databases/cistarget/motif2tf/motifs-v9-nr.hgnc-m0.001-o0.0.tbl'
        tracksDb = '/staging/leuven/res_00001/databases/cistarget/databases/homo_sapiens/hg38/refseq_r80/tc_v1/gene_based/encode_20190621__ChIP_seq_transcription_factor.hg38__refseq-r80__*feather'
        tracksAnnotation = '/staging/leuven/stg_00002/lcb/icistarget/data/annotations/homo_sapiens/hg38/track_annotations/encode_project_20190621__ChIP-seq_transcription_factor.homo_sapiens.hg38.bigwig_signal_pvalue.track_to_tf_in_motif_to_tf_format.tsv'
     }
     aucell {
        output = 'aucell_output.loom'
        rank_threshold = 5000
        auc_threshold = 0.05
        nes_threshold = 3.0
        min_genes_regulon = 0
        min_regulon_gene_occurrence = 0
     }
     numRuns = 2
     aggregate_features {
        use_chunking = true
        output_format = 'csv'
        compression = 'gzip'
     }
  }
  scope {
     genome = ''
     tree {
        level_1 = ''
        level_2 = ''
        level_3 = ''
     }
  }

} data { tenx { cellranger_mex = '/home/cheema/scenic_analysis/sample_data/outs' } } }

process { executor = 'local' withLabel:'compute_resources.*|compute_resourcesdefault' { cpus = 2 memory = '60 GB' time = '1h' clusterOptions = '-A cluster_account' } withLabel:compute_resourcesminimal { cpus = 1 memory = '1 GB' } withLabel:compute_resourcesmem { cpus = 4 memory = '160 GB' } withLabel:compute_resourcescpu { cpus = 20 memory = '80 GB' } withLabel:compute_resources__report { maxForks = 2 cpus = 1 memory = '160 GB' } withLabel:compute_resources24hqueue { time = '24h' } withLabel:'compute_resources__scenic.' { cpus = 4 memory = '60 GB' time = '24h' maxForks = 1 } withLabel:compute_resourcesscenic_grn { cpus = 4 memory = '120 GB' time = '24h' } withLabel:compute_resourcesscenic_cistarget { cpus = 4 memory = '80 GB' } withLabel:compute_resourcesscenic_aucell { cpus = 4 memory = '80 GB' } withLabel:'compute_resourcesscenic_multiruns.' { cpus = 4 memory = '60 GB' time = '24h' } withLabel:compute_resources__scenic_multiruns_motifs2regulons { memory = '60 GB' } }

timeline { enabled = true file = 'out/nextflow_reports/execution_timeline.html' }

report { enabled = true file = 'out/nextflow_reports/execution_report.html' }

trace { enabled = true file = 'out/nextflow_reports/execution_trace.txt' }

dag { enabled = true file = 'out/nextflow_reports/pipeline_dag.svg' }

min { enabled = false }

docker{ enabled = true autoMounts = true runOptions = '-B /ddn1/vol1/staging/leuven/stg_00002/,/staging/leuven/stg_00002/' }



The log file for nextflow is available here (https://pastebin.com/WSWUjZAd)
**Expected behavior**
I was expecting it to complete the analysis 

**Please complete the following information:**
 - OS: [Linux 20.04.1-Ubuntu SMP Wed Jan 19 14:08:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux]
 - Nextflow Version: [21.04.3]
 - vsn-pipelines Version: [e.g. 0.25.0]

Is there any way I can solve this problem?

Thanks,
MarcDalod commented 2 years ago

The error was due to the problem in config file generation, since I didn't choose docker while making config file so I was having errors. When I made the config file with the command given below, the error was removed. The correct way for making config file to be used by docker is given below,

nextflow config vib-singlecell-nf/vsn-pipelines \
      -profile tenx,single_sample_scenic,scenic_multiruns,scenic_use_cistarget_motifs,scenic_use_cistarget_tracks,hg38,singularity[ replace singularity with docker here] \
> pbmc10k.vsn-pipelines.complete.config

so If you choose docker then the last part is changed in the config file, specifically adopted for docker

docker{ enabled = true runOptions = '-i -v /home/cheema/your_user_name:/home/your_user_name' }