Deeplearning4j example TaskContextNotFoundException

[cbiehl]

When running the example for Deeplearning4j (DeepLearning4jDocumentTrainTest.java) I get the following exception(s): (I cloned the repository and ran the example as-is on Windows 10 in Eclipse Oxygen)

Update: Just checked on another Windows PC and it throws the same exception, might be a Windows issue.

INFO: Executing task [org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning] Feb 01, 2018 5:38:38 PM org.dkpro.lab.logging.impl.DefaultLoggingService error SEVERE: [Evaluation-DeepLearning-20180201173807116] Task failed [Evaluation-DeepLearning](caused by TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed.) Feb 01, 2018 5:38:38 PM org.dkpro.lab.logging.impl.DefaultLoggingService message INFO: [Evaluation-DeepLearning-20180201173807116] Shut down task Exception in thread "main" org.dkpro.lab.storage.UnresolvedImportException: -Unable to resolve import of task [org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning/preprocessorOutputTrain]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed. -Unable to resolve import of task [org.dkpro.tc.core.task.deep.EmbeddingTask-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning/output]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning] has never been executed. -Unable to resolve import of task [org.dkpro.tc.core.task.deep.VectorizationTask-Train-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning/preprocessorOutputTrain]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed. -Unable to resolve import of task [org.dkpro.tc.core.task.deep.VectorizationTask-Test-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.InitTaskDeep-Test-DeepLearning/preprocessorOutputTest]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Test-DeepLearning] has never been executed. -Unable to resolve import of task [org.dkpro.tc.ml.deeplearning4j.Deeplearning4jTestTask-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.VectorizationTask-Test-DeepLearning/output]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.VectorizationTask-Test-DeepLearning] has never been executed.; nested exception is org.dkpro.lab.storage.UnresolvedImportException: Unable to resolve import of task [org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning/preprocessorOutputTrain]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed. at org.dkpro.lab.engine.impl.BatchTaskEngine.executeConfiguration(BatchTaskEngine.java:269) at org.dkpro.lab.engine.impl.BatchTaskEngine.run(BatchTaskEngine.java:133) at org.dkpro.lab.engine.impl.DefaultTaskExecutionService.run(DefaultTaskExecutionService.java:52) at org.dkpro.lab.Lab.run(Lab.java:113) at org.dkpro.tc.examples.deeplearning.dl4j.doc.DeepLearning4jDocumentTrainTest.runTrainTest(DeepLearning4jDocumentTrainTest.java:111) at org.dkpro.tc.examples.deeplearning.dl4j.doc.DeepLearning4jDocumentTrainTest.main(DeepLearning4jDocumentTrainTest.java:60) Caused by: org.dkpro.lab.storage.UnresolvedImportException: Unable to resolve import of task [org.dkpro.tc.core.task.deep.PreparationTask-DeepLearning] pointing to [task-latest://org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning/preprocessorOutputTrain]; nested exception is org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed. at org.dkpro.lab.engine.impl.BatchTaskEngine$ScopedTaskContext.resolve(BatchTaskEngine.java:559) at org.dkpro.lab.engine.impl.DefaultTaskContextFactory.resolveImports(DefaultTaskContextFactory.java:151) at org.dkpro.lab.engine.impl.DefaultTaskContextFactory.createContext(DefaultTaskContextFactory.java:105) at org.dkpro.lab.uima.engine.simple.SimpleExecutionEngine.run(SimpleExecutionEngine.java:77) at org.dkpro.lab.engine.impl.BatchTaskEngine.runNewExecution(BatchTaskEngine.java:346) at org.dkpro.lab.engine.impl.BatchTaskEngine.executeConfiguration(BatchTaskEngine.java:241) ... 5 more Caused by: org.dkpro.lab.storage.TaskContextNotFoundException: Task [org.dkpro.tc.core.task.deep.InitTaskDeep-Train-DeepLearning] has never been executed. at org.dkpro.lab.engine.impl.ImportUtil.createContextNotFoundException(ImportUtil.java:124) at org.dkpro.lab.engine.impl.BatchTaskEngine.getLatestExecution(BatchTaskEngine.java:324) at org.dkpro.lab.engine.impl.BatchTaskEngine.access$000(BatchTaskEngine.java:65) at org.dkpro.lab.engine.impl.BatchTaskEngine$ScopedTaskContext.resolve(BatchTaskEngine.java:556) ... 10 more

[Horsmann]

Hi, thank you for reporting. I can't reproduce the error on my machine (OSX) and Jenkins (Linux) is working fine, too. I also tried running TC in Oxygen without problems. This makes this error hard to track.

@cbiehl Can you run none of the examples or do just the DL4j example crash? The background setup is different for deep learning, so, could you try one of the non-deep-learning examples and see if you get the same error?

@reckart Do you maybe have an idea what could cause this problem?

[reckart]

Maybe a path problem, e.g. a space character or special character in the path to where DKPro Lab/TC stores the contexts?

[reckart]

Could also be a slash/backslash issue.

[cbiehl]

Hi @Horsmann, hi @reckart,

thanks for the quick response! I get the same exception when running the keras document classification example, but e.g. the LibsvmTwentyNewsgroups.java example and the WekaTwitterSentimentDemo.java example run without issues.

I don't have spaces in the path to the cloned repo, but I'm pretty sure this is a Windows issue (since the examples worked on multiple Linux machines). Guess I'll just use a proper OS :-)

Let me know if you need more information. The issue seems to be reproducible on other Windows PCs.

[Horsmann]

@cbiehl I think I found the problem. The demo uses a relative file path for the word embeddings, which points to "src/test..". This relative path is resolved internally to the full file path on Windows, i.e. C:/.../src/... - TC tries to find the key, the relative path, but finds only the absolute one. This causes confusion in the execution of the different tasks/steps letting TC believe a task was not executed.

Long story short, if you provide the absolute file path as parameter this error should not show up again. At the moment all DL demos have this problem on Window as either the embedding or the framework-code-snippet are provided as relative paths.

Sorry for the inconvenience but I hope you can execute the tests now, just don't use relative paths for providing embeddings/code-snippets. I have to think about it how to avoid this error in the future.

btw. there is a follow up issue in the assert-statements of the DL4j test, the windows line-break is not trimmed properly. Leading to a test-case failure. I think I will have to run a Windows-Jenkins to avoid this kind of errors in the future :/.

[reckart]

@Horsmann We have a Windows Jenkins at UKP. If you want, I can set up a DKPro TC / Lab build job on it.

[Horsmann]

@reckart Yes, please, this would be helpful :)

[reckart]

Done.

[Horsmann]

@reckart thanks!

@cbiehl Could you pull again and give it another try. I think I fixed the file-path issue. Its working in my Win 7 VM at least.

[Horsmann]

@reckart I have to update the window-binaries of SvmHmm. The windows binaries still had the .exe suffix and are consequently not found when the RuntimeProvider tries to load them without this suffix.

I updated the installation script of the binares in the last commit to rename the binaries for windows and drop the .exe suffix https://github.com/dkpro/dkpro-tc/commit/3f2a719bd66df1f71892d5ebab9736a8e84b246c

Could you delete the old binary-jar and update the jar with the new one, the installation script to build the new version is referenced in the commit

[reckart]

The new JAR should have a new version such that Maven downloads it. Otherwise, if there is already a local copy of the JAR Maven will not fetch it.

[Horsmann]

@reckart I uploaded a new version of the binary to the ukp-staging repository but I am doing something wrong; the binaries are not found. When installing them manually via the script it works. Did I deploy them wrongly?

https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-ukp-staging-local/org/dkpro/tc/ml/svmhmm-bin/org.dkpro.tc.ml.svmhmm-bin/

I tried again with an increased artifact number but somehow Jenkins is either not using the repository or the artifacts are not found

[Horsmann]

@reckart is there some secret rule which repositories are used during model building it looks on the command line like maven is not even trying to use the staging repository to retrieve the new binary. If I delete all other repositories with only staging left, it says eventually that the binary isn't found, which I don't understand either. I uploaded two binaries to the staging repository of UKP because I can't delete btw.

[cbiehl]

@Horsmann it works on my Windows machine too now, thanks for the quick fix!

[reckart]

Well, Maven uses the repos defined in the pom(s) or in the settings.xml. If you have defined a catch-all mirror in the settings.xml, then the repos from the POMs are ignored.

Do you have a settings.xml with a catch-all mirror?

<mirrorOf>*,...</mirrorOf>

[Horsmann]

No, i use the default one on the dkpro website.

I declare the repository in the TC parent pom like this:

    <repository>
        <id>central</id>
        <name>ukp-oss-staging</name>
        <url>https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-ukp-staging-local</url>
        <releases>
        <enabled>true</enabled>
      </releases>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>

but the artifact is not downloaded.

When I run mvn install from command line with the -X switch I see that the repo seems to be recognized in general but nothings seems to be found there

[DEBUG] Failure to find org.dkpro.tc:dkpro-tc-core:1.0.0-SNAPSHOT/maven-metadata.xml in https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-ukp-staging-local was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced```

Afterwards the release repo is attempted where nothing is found and the build fails

[Horsmann]

@cbiehl great :)

[reckart]

Try "mvn -U clean install"?

[Horsmann]

I did already. no effect. The file in the repository is for some reason not found :/

[reckart]

Well... the last error message you have posted is about a dkpro-tc-core SNAPSHOT version - that certainly is not in the staging repo. Add back the SNAPSHOTs repo, then try again, otherwise you have one problem shadowing another one.

[Horsmann]

Well, I don't know what is wrong and I ran out of ideas, too. Could you add the binary ?

[reckart]

The one with version "20080814.3"?

[Horsmann]

The current version is .1. I forget to check the checkbox when deploying the .2, the binaries are the same in the .3 its just this meta-data file or what it is that has not automatically been created for the .2 file. Maybe it is the cleanest way if you just run the script in the SVMHMM project again to install a correct .2 version?

[reckart]

Ok, I've added it to my todo list.

[Horsmann]

@reckart did you had time for the new binary yet :/?

[reckart]

Ok, here you are: https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-releases/org/dkpro/tc/org.dkpro.tc.ml.svmhmm-bin/20080814.2/

[Horsmann]

Thank you.

[Horsmann]

@reckart Hm, did not work out the way I expected. Windows is complaining that the new binaries are

not recognized as an internal or external command, operable program or batch file

If I rename the binaries to use the .exe suffix again they start working again (if executed manually, not in TC). Any idea what is causing the problem?

[reckart]

Are you using the DKPro Core RuntimeProvider with a manifest file that tells the provider to mark the executable as such?

[Horsmann]

I think I found it, a new problem surfaced. SvmHmm has a weird filename restriction of 255 characters. Any longer paths lead to an error, which seemed to cause the problem in this case.