pditommaso
commented
4 years ago This looks something odd, in the metadata generated by the pipeline (a misplaced closure in a tag directive?).
Closed dmorais closed 3 years ago
pditommaso
commented
4 years ago This looks something odd, in the metadata generated by the pipeline (a misplaced closure in a tag directive?).
dweemx
commented
4 years ago Hi @dmorais , How does your config look like ?
dmorais
commented
4 years ago @dweemx That's what the config looks like. Basically the config of the example ran with the code below. The only change was on the singularity mounting point line ( I had to replace ',' by ":"
nextflow config vib-singlecell-nf/vsn-pipelines \ -profile tenx,singularity,single_sample > single_sample.config
manifest {
name = 'vib-singlecell-nf/vsn-pipelines'
description = 'A repository of pipelines for single-cell data in Nextflow DSL2'
homePage = 'https://github.com/vib-singlecell-nf/vsn-pipelines'
version = '0.19.0'
mainScript = 'main.nf'
defaultBranch = 'master'
nextflowVersion = '!20.04.1'
}
params {
global {
project_name = '10x_PBMC'
outdir = 'out'
qsubaccount = ''
qsubemail = ''
}
misc {
test {
enabled = false
}
}
sc {
scanpy {
container = 'vibsinglecellnf/scanpy:0.5.0'
report {
annotations_to_plot = []
}
feature_selection {
report_ipynb = '/src/scanpy/bin/reports/sc_select_variable_genes_report.ipynb'
method = 'mean_disp_plot'
minMean = 0.0125
maxMean = 3
minDisp = 0.5
off = 'h5ad'
}
feature_scaling {
method = 'zscore_scale'
maxSD = 10
off = 'h5ad'
}
neighborhood_graph {
nPcs = 50
off = 'h5ad'
}
dim_reduction {
report_ipynb = '/src/scanpy/bin/reports/sc_dim_reduction_report.ipynb'
pca {
method = 'pca'
nComps = 50
off = 'h5ad'
}
umap {
method = 'umap'
off = 'h5ad'
}
tsne {
method = 'tsne'
nJobs = 10
off = 'h5ad'
}
}
clustering {
report_ipynb = '/src/scanpy/bin/reports/sc_clustering_report.ipynb'
method = 'louvain'
resolution = 0.8
off = 'h5ad'
}
marker_genes {
method = 'wilcoxon'
ngenes = 0
groupby = 'louvain'
off = 'h5ad'
}
filter {
report_ipynb = '/src/scanpy/bin/reports/sc_filter_qc_report.ipynb'
cellFilterMinNGenes = 200
cellFilterMaxNGenes = 4000
cellFilterMaxPercentMito = 0.15
geneFilterMinNCells = 3
off = 'h5ad'
outdir = 'out'
}
data_transformation {
method = 'log1p'
off = 'h5ad'
}
normalization {
method = 'cpx'
countsPerCellAfter = 10000
off = 'h5ad'
}
}
scope {
genome = ''
tree {
level_1 = ''
level_2 = ''
level_3 = ''
}
}
file_converter {
tagCellWithSampleId = true
useFilteredMatrix = true
makeVarIndexUnique = false
}
}
data {
tenx {
cellranger_mex = 'single_sample/data/10x/1k_pbmc/1k_pbmc_v2_chemistry/outs/'
}
}
utils {
container = 'vibsinglecellnf/utils:0.3.0'
workflow_configuration {
report_ipynb = '/src/utils/bin/reports/workflow_configuration_template.ipynb'
}
publish {
compressionLevel = 6
}
}
parseConfig = { sample, paramsGlobal, paramsLocal ->
def lv = { a,b -> return org.codehaus.groovy.runtime.MethodRankHelper.delDistance(a, b) }
def pL = paramsLocal.collectEntries { k,v ->
if (v instanceof Map) {
if (v.containsKey(sample))
return [k, v[sample]]
if (v.containsKey('default'))
return [k, v['default']]
def closeMatches = v.collectEntries { vk, vv -> [lv(vk, sample), vk] }.keySet().findAll { it < 30}
if(closeMatches.size() > 0)
throw new Exception("The sample " + sample + " is not found in " + v +" ; Make sure your samples are correctly specified when using the multi-sample feature.")
else
return [k,v]
} else {
return [k,v]
}
}
return [global: paramsGlobal, local: pL]
}
}
process {
executor = 'local'
withLabel:qsub {
executor = 'pbs'
}
withLabel:local {
executor = 'local'
}
}
timeline {
enabled = true
file = 'out/nextflow_reports/execution_timeline.html'
}
report {
enabled = true
file = 'out/nextflow_reports/execution_report.html'
}
trace {
enabled = true
file = 'out/nextflow_reports/execution_trace.txt'
}
dag {
enabled = true
file = 'out/nextflow_reports/pipeline_dag.svg'
}
min {
enabled = false
}
singularity {
enabled = true
autoMounts = true
runOptions = '-B /home/dmorais/projects/nextflow/vib-singlecell/stg_00002/:/staging/leuven/stg_00002/'
}@dmorais , We were able to reproduce your error. We have probably nailed down the issue: we are using a closure in the config. @pditommaso can you confirm that this could cause the problem ? It's been a while we are using this and it's working fine with Nextflow but Nf Tower doesn't seem to like somwhow. Is this something possible to fix ?
dmorais
commented
4 years ago @dweemx Thanks. Hopefully, the fix won't be too hard. I appreciate your help.
Thanks @pditommaso for looking into it as well.
pditommaso
commented
3 years ago Hard to say what's wrong in your config, but as of nextflow 20.09.1-edge it reports a warning instead of crashing. It may help to identify the problem.
Closing this issue.
When I try to run vib-singlecell-nf /vsn-pipelines example ( the single-sample) configured to use singularity I get this error
_Invalid method
doCallinvocation with arguments: on closure15 typeThe pipeline without the -with-tower works just fine, but when I add this argument it throws an error. I also tried to add a block of code inside the config, as suggested in the tower documentation, but I get the same error.
Below is the Stack trace