search

nextflow-io / nf-validation

Params validation plugin for Nextflow pipelines
https://nextflow-io.github.io/nf-validation/
Apache License 2.0
48 stars 20 forks source link

ImmutableMap clashes with the `-resume` flag #47

Closed nvnieuwk closed 1 year ago

nvnieuwk commented 1 year ago

When resuming a pipeline with ImmutableMap meta, following warnings are given:

WARN: [CMGG_CMGGSTRUCTURAL:CMGGSTRUCTURAL:BAM_REPEAT_ESTIMATION_EXPANSIONHUNTER:EXPANSIONHUNTER (1)] Unable to resume cached task -- See log file for details
WARN: [CMGG_CMGGSTRUCTURAL:CMGGSTRUCTURAL:BAM_REPEAT_ESTIMATION_EXPANSIONHUNTER:EXPANSIONHUNTER (4)] Unable to resume cached task -- See log file for details
WARN: [CMGG_CMGGSTRUCTURAL:CMGGSTRUCTURAL:BAM_REPEAT_ESTIMATION_EXPANSIONHUNTER:EXPANSIONHUNTER (2)] Unable to resume cached task -- See log file for details
WARN: [CMGG_CMGGSTRUCTURAL:CMGGSTRUCTURAL:BAM_REPEAT_ESTIMATION_EXPANSIONHUNTER:EXPANSIONHUNTER (3)] Unable to resume cached task -- See log file for details
WARN: [CMGG_CMGGSTRUCTURAL:CMGGSTRUCTURAL:BAM_REPEAT_ESTIMATION_EXPANSIONHUNTER:EXPANSIONHUNTER (5)] Unable to resume cached task -- See log file for details

With this as error in .nextflow.log:

com.esotericsoftware.kryo.KryoException: Unable to find class: nextflow.validation.ImmutableMap
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:138)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:115)
    at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:752)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:143)
    at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:21)
    at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
    at com.esotericsoftware.kryo.Kryo$readClassAndObject$8.call(Unknown Source)
    at nextflow.util.KryoHelper.deserialize(SerializationHelper.groovy:181)
    at nextflow.util.KryoHelper.deserialize(SerializationHelper.groovy)
    at nextflow.util.KryoHelper$deserialize.call(Unknown Source)
    at nextflow.processor.TaskContext.deserialize(TaskContext.groovy:202)
    at nextflow.cache.CacheDB.getTaskEntry(CacheDB.groovy:88)
    at nextflow.processor.TaskProcessor.checkCachedOrLaunchTask(TaskProcessor.groovy:770)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:48)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:189)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:57)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:203)
    at nextflow.processor.TaskProcessor.invokeTask(TaskProcessor.groovy:618)
    at nextflow.processor.InvokeTaskAdapter.call(InvokeTaskAdapter.groovy:52)
    at groovyx.gpars.dataflow.operator.DataflowOperatorActor.startTask(DataflowOperatorActor.java:120)
    at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor.access$001(ForkingDataflowOperatorActor.java:35)
    at groovyx.gpars.dataflow.operator.ForkingDataflowOperatorActor$1.run(ForkingDataflowOperatorActor.java:58)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.ClassNotFoundException: nextflow.validation.ImmutableMap
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:467)
    at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:136)
    ... 29 common frames omitted

Possible solution: try to find a way to make ImmutableMap available to the main flow

nvnieuwk commented 1 year ago

Issue opened in the main nextflow repo: https://github.com/nextflow-io/nf-schema/issues/23

nvnieuwk commented 1 year ago

I would suggest turning of the use of ImmutableMap for now until this is fixed, you can still use it when running production code that does not need to be resumed