search

nf-core / differentialabundance

Differential abundance analysis for feature/ observation matrices from platforms such as RNA-seq
https://nf-co.re/differentialabundance
MIT License
64 stars 37 forks source link

PosixFileAttributeView #263

Open nick-youngblut opened 7 months ago

nick-youngblut commented 7 months ago

Description of the bug

I'm running the pipeline with GCP Batch. My GCP profile:

  gcp {
    process.executor       = "google-batch"
    process.errorStrategy  = "retry"
    process.maxRetries     = 3
    params.max_cpus        = 16
    params.max_memory      = "128.GB"
    params.max_time        = "96.h"
    google.project         = "arc-genomics"
    google.location        = "us-west1"
    fusion.enabled         = true
    wave.enabled           = true
    process.scratch        = false
  }

It appears that PLOT_EXPLORATORY is trying to use an attribute view that this not supported.

The error:

[e7/f55d34] Cached process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:GTF_TO_TABLE (genes)
[ba/c482a3] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:VALIDATOR (samples.csv)
[23/2750fc] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:CUSTOM_MATRIXFILTER ([id:study])
[14/4084dd] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL ([id:time_series, variable:condition, reference:control, target:treatment, blocking:replicate])
[b7/fb11ce] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_NORM ([id:time_series, variable:condition, reference:control, target:treatment, blocking:replicate])
[58/00dc1a] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_EXPLORATORY (condition)
ERROR ~ PosixFileAttributeView

 -- Check '.nextflow.log' file for details

The relevant section from the nextflow.log:

Apr-12 18:41:31.762 [Task submitter] INFO  nextflow.Session - [58/00dc1a] Submitted process > NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:PLOT_EXPLORATORY (condition)
Apr-12 18:41:36.838 [Task monitor] DEBUG n.c.g.batch.GoogleBatchTaskHandler - [GOOGLE BATCH] Process `NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL ([id:time_series, variable:condition, reference:control, target:treatment, blocking:replicate])` - terminated job=nf-144084dd-1712947138681; state=SUCCEEDED
Apr-12 18:41:36.923 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL ([id:time_series, variable:condition, reference:control, target:treatment, blocking:replicate]); status: COMPLETED; exit: 0; error: -; workDir: gs://arc-genomics-nextflow/differentialabundance/work/14/4084dd744f455abfc18a0834878f56]
Apr-12 18:41:40.881 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Process NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:DESEQ2_DIFFERENTIAL > Skipping output binding because one or more optional files are missing: fileoutparam<5:1>
Apr-12 18:41:41.075 [Actor Thread 217] ERROR nextflow.extension.OperatorImpl - @unknown
java.lang.UnsupportedOperationException: PosixFileAttributeView
    at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.getFileAttributeView(CloudStorageFileSystemProvider.java:855)
    at java.base/java.nio.file.Files.getFileAttributeView(Files.java:1790)
    at java.base/java.nio.file.CopyMoveHelper.copyToForeignTarget(CopyMoveHelper.java:151)
    at java.base/java.nio.file.CopyMoveHelper.moveToForeignTarget(CopyMoveHelper.java:193)
    at java.base/java.nio.file.Files.move(Files.java:1433)
    at nextflow.file.SimpleFileCollector.saveFile(SimpleFileCollector.groovy:102)
    at nextflow.file.FileCollector.saveTo0(FileCollector.groovy:228)
    at nextflow.file.FileCollector.saveTo(FileCollector.groovy:251)
    at nextflow.file.FileCollector$saveTo.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
    at nextflow.extension.CollectFileOp.emitItems(CollectFileOp.groovy:187)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
    at groovy.lang.MetaClassImpl.invokeMethodClosure(MetaClassImpl.java:1042)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1128)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
    at groovy.lang.Closure.call(Closure.java:427)
    at groovy.lang.Closure.call(Closure.java:416)
    at groovy.lang.Closure$call.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:53)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:139)
    at nextflow.extension.DataflowHelper$3.afterStop(DataflowHelper.groovy:257)
    at groovyx.gpars.dataflow.operator.DataflowProcessor.fireAfterStop(DataflowProcessor.java:324)
    at groovyx.gpars.dataflow.operator.DataflowProcessorActor.afterStop(DataflowProcessorActor.java:59)
    at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.base/java.lang.reflect.Method.invoke(Method.java:580)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:107)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:323)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1254)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1030)
    at org.codehaus.groovy.runtime.InvokerHelper.invokePojoMethod(InvokerHelper.java:1024)
    at org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:1015)
    at groovyx.gpars.actor.Actor.callDynamic(Actor.java:369)
    at groovyx.gpars.actor.Actor.handleTermination(Actor.java:320)
    at groovyx.gpars.actor.AbstractLoopingActor.terminate(AbstractLoopingActor.java:204)
    at groovyx.gpars.dataflow.operator.DataflowProcessor.terminate(DataflowProcessor.java:147)
    at groovyx.gpars.dataflow.operator.DataflowProcessorActor.checkPoison(DataflowProcessorActor.java:115)
    at groovyx.gpars.dataflow.operator.DataflowOperatorActor.onMessage(DataflowOperatorActor.java:83)
    at groovyx.gpars.actor.impl.SDAClosure$1.call(SDAClosure.java:43)
    at groovyx.gpars.actor.AbstractLoopingActor.runEnhancedWithoutRepliesOnMessages(AbstractLoopingActor.java:293)
    at groovyx.gpars.actor.AbstractLoopingActor.access$400(AbstractLoopingActor.java:30)
    at groovyx.gpars.actor.AbstractLoopingActor$1.handleMessage(AbstractLoopingActor.java:93)
    at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:132)
    at java.base/java.util.concurrent.ThreadPerTaskExecutor$TaskRunner.run(ThreadPerTaskExecutor.java:314)
    at java.base/java.lang.Thread.run(Thread.java:1583)
Apr-12 18:41:41.082 [Actor Thread 217] DEBUG nextflow.Session - Session aborted -- Cause: PosixFileAttributeView
Apr-12 18:41:41.097 [Actor Thread 217] DEBUG nextflow.Session - The following nodes are still active:
[process] NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:CUSTOM_DUMPSOFTWAREVERSIONS
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: versions
  port 1: (cntrl) -     ; channel: $

[process] NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:SHINYNGS_APP
  status=ACTIVE
  port 0: (value) bound ; channel: -
  port 1: (queue) OPEN  ; channel: -
  port 2: (value) bound ; channel: contrast_stats_assay
  port 3: (cntrl) -     ; channel: $

[process] NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:RMARKDOWNNOTEBOOK
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (queue) OPEN  ; channel: parameters
  port 2: (queue) OPEN  ; channel: input_files
  port 3: (cntrl) -     ; channel: $

[process] NFCORE_DIFFERENTIALABUNDANCE:DIFFERENTIALABUNDANCE:MAKE_REPORT_BUNDLE
  status=ACTIVE
  port 0: (queue) OPEN  ; channel: -
  port 1: (cntrl) -     ; channel: $

I'm guessing it has something to do with the plotting in the PLOT_EXPLORATORY R code.

Command used and terminal output

I'm running the pipeline on output from the `nf-core/rnaseq` pipeline. 
My command:

nextflow run main.nf \
   -ansi-log false \
   -profile docker,gcp \
   -work-dir gs://arc-genomics-nextflow/differentialabundance/work \
   --gtf gs://arc-genomics-references/cellranger/refdata-gex-GRCh38-2024-A/genes/genes.gtf \
   --matrix gs://arc-genomics-nextflow/bulk-rna/KL-NPC-Differentiation/star_salmon/salmon.merged.gene_counts.tsv \
   --transcript_length_matrix gs://arc-genomics-nextflow/bulk-rna/KL-NPC-Differentiation/star_salmon/salmon.merged.gene_lengths.tsv \
   --input gs://arc-genomics-nextflow/bulk-rna/KL-NPC-Differentiation/diff-abund/samples.csv \
   --contrasts gs://arc-genomics-nextflow/bulk-rna/KL-NPC-Differentiation/diff-abund/contrasts.csv \
   --outdir gs://arc-genomics-nextflow/bulk-rna/KL-NPC-Differentiation/diff-abund

Relevant files

No response

System information