MMU - Hackathon - Githubissues

mattshardlow commented 6 years ago

Currently we have developed an initial set of components for the text mining for journalism project. These are available via github here: https://github.com/MMU-TDMLab/TextMiningForJournalism

Our next step is to mavenise these components and register them with the platform.

We have had several delays with processing the funding and contract between BSC and MMU, which means we are currently behind schedule.

My feeling is that we would benefit more from the hackathon at a later point (2-3 weeks time), when we have sorted the funding and have had chance to attempt to register and run our components / application.

pennyl67 commented 6 years ago

@reckart done at guidelines; should we add in any other place in the documentation of github?

reckart commented 6 years ago

@pennyl67 I think the guidelines is sufficient.

mattshardlow commented 6 years ago

I've now updated the UIMA versions throughout my components and released new artefacts to maven central. (I'm waiting for them to filter through).

I've updated the metadata of my component using the form at test.openminted.eu. However,changing the location under the 'distribution' section has only affected the componentInfo->distributionInfos->componentDistributionInfo->distributionLocation field in the metadata. This is now out of sync with the componentInfo->identificationInfo->resourceIdentifiers->resourceIdentifier field, which still has the old version (0.0.1).

Is there any way of forcing this change on the uploaded version of the component's metadata? Or do I need to start the process again?

gkirtzou commented 6 years ago

I think that you need to re-register components from scratch in order to retrieve the new maven project, generate the docker image and the respective galaxy wrappers.

mattshardlow commented 6 years ago

ok, will give it a go

mattshardlow commented 6 years ago

Tried that, but it says 'already added'. Does this mean that it's not possible to upload a new version of a component to the framework if an older version has already been registered?

mattshardlow commented 6 years ago

ah, ok maybe I misinterpreted the message. Apparently it's added it to the 'my components' section (3 times!)

mattshardlow commented 6 years ago

I've now registered the component. But when I try to create a new workflow, I don't have the option to create a workflow using v2. There is a button in the galaxy interface to switch to version 0.0.2, but this doesn't appear to do anything. Is it disabled? One option would be to delete v0.0.1. Is that the right way to go about this?

mattshardlow commented 6 years ago

I tried deleting the older version from 'my components' and runnning an updated workflow with the new version, but it errored again. Please can you send me the log, in case there's anything helpful in there. Any support you can offer to get this running is appreciated (especially given the short deadline to the launch).

gkirtzou commented 6 years ago

Here is the log from your component

Using default installation dir /opt/omtd-component-executor
ocoord:uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.1
15:58:31.277 [main] INFO eu.openminted.workflows.componentdependencies.DependenciesFetcherMain - ...

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.2.RELEASE)

2018-05-11 15:58:35.904  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : Starting DependenciesFetcherMain v0.0.1-SNAPSHOT on snf-774931 with PID 16 (/opt/omtd-component-executor/omtd-component-dependencies-fetcher/target/omtd-component-dependencies-fetcher-0.0.1-SNAPSHOT-exec.jar started by root in /opt/omtd-component-executor)
2018-05-11 15:58:35.911  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : No active profile set, falling back to default profiles: default
2018-05-11 15:58:36.398  INFO 16 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@22d8cfe0: startup date [Fri May 11 15:58:36 UTC 2018]; root of context hierarchy
2018-05-11 15:58:40.296  INFO 16 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-05-11 15:58:40.351  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.1
classPathListFileName:./TDMClasspathLists/classpath.uk.ac.mmu.tdmlab.journalism_TextMiningForJournalismApplication_0.0.1
uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.1 

length:72
FIlTER/opt/TDMlocalRepo/org/springframework/spring-core/3.2.16.RELEASE/spring-core-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-context/3.2.16.RELEASE/spring-context-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-aop/3.2.16.RELEASE/spring-aop-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-expression/3.2.16.RELEASE/spring-expression-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-beans/3.2.16.RELEASE/spring-beans-3.2.16.RELEASE.jar

2018-05-11 15:59:38.663  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : Started DependenciesFetcherMain in 65.993 seconds (JVM running for 69.581)
2018-05-11 15:59:38.664  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : DONE!
2018-05-11 15:59:38.667  INFO 16 --- [       Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@22d8cfe0: startup date [Fri May 11 15:58:36 UTC 2018]; root of context hierarchy
2018-05-11 15:59:38.686  INFO 16 --- [       Thread-1] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
coordinates:uk.ac.mmu.tdmlab.journalism_TextMiningForJournalismApplication_0.0.1
inDir:tmp
otDir:/srv/galaxy/database/jobs_directory/001/1408/working/out/
uimaParams:
15:59:41.524 [main] INFO eu.openminted.workflows.uima.executor.PipelineCommandLineRunner - ...

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.2.RELEASE)

2018-05-11 15:59:46.179  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : Starting PipelineCommandLineRunner v0.0.1-SNAPSHOT on snf-774931 with PID 45 (/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar started by root in /opt/omtd-component-executor/scripts/working)
2018-05-11 15:59:46.182  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : No active profile set, falling back to default profiles: default
2018-05-11 15:59:46.656  INFO 45 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@6e5e91e4: startup date [Fri May 11 15:59:46 UTC 2018]; root of context hierarchy
2018-05-11 15:59:49.687  INFO 45 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2018-05-11 15:59:50.648  INFO 45 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
parsing...
input tmp
output /srv/galaxy/database/jobs_directory/001/1408/working/out/
className uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication
2018-05-11 15:59:50.735  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : 

Executing..
========================
className:uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication
inputDir:tmp
outputDir:/srv/galaxy/database/jobs_directory/001/1408/working/out/
**parameters
========================
========================

2018-05-11 15:59:52.358  WARN 45 --- [           main] org.apache.uima                          : SAXTransformerFactory didn't recognize setting attribute http://javax.xml.XMLConstants/property/accessExternalDTD
2018-05-11 15:59:52.359  WARN 45 --- [           main] org.apache.uima                          : SAXTransformerFactory didn't recognize setting attribute http://javax.xml.XMLConstants/property/accessExternalStylesheet
2018-05-11 15:59:53.130  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:53.257  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:53.348  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:53.471  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 15:59:53.544  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:53.550  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:53.948  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:54.081  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "lenient" in context / = "false" 
2018-05-11 15:59:54.093  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "sourceLocation" in context / = "tmp" 
2018-05-11 15:59:54.097  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "patterns" in context / = "[[+]**/*.xmi]" 
2018-05-11 15:59:54.103  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "useDefaultExcludes" in context / = "true" 
2018-05-11 15:59:54.105  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "includeHidden" in context / = "false" 
2018-05-11 15:59:54.144  INFO 45 --- [           main] d.t.ukp.dkpro.core.io.xmi.XmiReader      : Scanning [file:/opt/omtd-component-executor/scripts/working/tmp/]
2018-05-11 15:59:54.173  INFO 45 --- [           main] d.t.ukp.dkpro.core.io.xmi.XmiReader      : Found [0] resources to be read
2018-05-11 15:59:54.495  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:54.498  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:54.520  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:54.542  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 15:59:54.565  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:54.567  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:54.598  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:54.647  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:54.681  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 15:59:54.849  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication] initialization begin
2018-05-11 15:59:55.328  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:55.361  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:55.396  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 15:59:55.449  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 15:59:55.463  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 15:59:55.990  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.StanfordNLPTagger] initialization begin
2018-05-11 15:59:56.540  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator tokenize
2018-05-11 15:59:56.597  INFO 45 --- [           main] e.s.nlp.pipeline.TokenizerAnnotator      : No tokenizer type provided. Defaulting to PTBTokenizer.
2018-05-11 15:59:56.684  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator ssplit
2018-05-11 15:59:56.713  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator pos
2018-05-11 16:00:01.390  INFO 45 --- [           main] e.s.nlp.tagger.maxent.MaxentTagger       : Loading POS tagger from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [4.6 sec].
2018-05-11 16:00:01.395  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator lemma
2018-05-11 16:00:01.401  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator ner
2018-05-11 16:00:04.815  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [3.1 sec].
2018-05-11 16:00:10.764  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [5.9 sec].
2018-05-11 16:00:12.024  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [1.3 sec].
2018-05-11 16:00:12.034  INFO 45 --- [           main] edu.stanford.nlp.time.JollyDayHolidays   : Initializing JollyDayHoliday for SUTime from classpath edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.
2018-05-11 16:00:12.729  INFO 45 --- [           main] e.s.n.time.TimeExpressionExtractorImpl   : Using following SUTime rules: edu/stanford/nlp/models/sutime/defs.sutime.txt,edu/stanford/nlp/models/sutime/english.sutime.txt,edu/stanford/nlp/models/sutime/english.holidays.sutime.txt
2018-05-11 16:00:28.215  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 580641 unique entries out of 581790 from edu/stanford/nlp/models/kbp/regexner_caseless.tab, 0 TokensRegex patterns.
2018-05-11 16:00:28.372  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 4857 unique entries out of 4868 from edu/stanford/nlp/models/kbp/regexner_cased.tab, 0 TokensRegex patterns.
2018-05-11 16:00:28.376  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 585498 unique entries from 2 files
2018-05-11 16:01:04.828  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.StanfordNLPTagger] initialization successful
2018-05-11 16:01:05.015  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 16:01:05.016  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 16:01:05.017  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 16:01:05.024  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhereAnnotator] initialization begin
2018-05-11 16:01:05.030  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhereAnnotator] initialization successful
2018-05-11 16:01:05.136  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 16:01:05.138  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 16:01:05.139  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 16:01:05.142  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhoAnnotator] initialization begin
2018-05-11 16:01:05.144  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhoAnnotator] initialization successful
2018-05-11 16:01:05.288  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 16:01:05.290  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 16:01:05.291  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 16:01:05.296  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhenAnnotator] initialization begin
2018-05-11 16:01:05.298  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.WhenAnnotator] initialization successful
2018-05-11 16:01:05.302  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication] initialization successful
2018-05-11 16:01:05.469  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 16:01:05.471  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 16:01:05.483  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 16:01:12.745  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.8.0.jar!/desc/type/LexicalUnits.xml
2018-05-11 16:01:12.767  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/POS.xml
2018-05-11 16:01:12.786  INFO 45 --- [           main] o.a.u.r.metadata.impl.Import_impl        : Import by name: jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.8.0.jar!/desc/type/Morpheme.xml
2018-05-11 16:01:12.796  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 16:01:12.818  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 16:01:12.822  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 16:01:12.825  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "targetLocation" in context / = "/srv/galaxy/database/jobs_directory/001/1408/working/out/" 
2018-05-11 16:01:12.826  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "singularTarget" in context / = "false" 
2018-05-11 16:01:12.827  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "compression" in context / = "NONE" 
2018-05-11 16:01:12.827  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "stripExtension" in context / = "false" 
2018-05-11 16:01:12.828  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "useDocumentId" in context / = "false" 
2018-05-11 16:01:12.828  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "escapeDocumentId" in context / = "true" 
2018-05-11 16:01:12.829  INFO 45 --- [           main] o.a.u.r.impl.ConfigurationManager_impl   : Parameter "overwrite" in context / = "true" 
2018-05-11 16:01:12.841  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [de.tudarmstadt.ukp.dkpro.core.io.xmi.XmiWriter] initialization begin
2018-05-11 16:01:12.909  INFO 45 --- [           main] o.a.u.a.i.PrimitiveAnalysisEngine_impl   : AnalysisEngine [de.tudarmstadt.ukp.dkpro.core.io.xmi.XmiWriter] initialization successful
2018-05-11 16:01:12.921  WARN 45 --- [           main] org.apache.uima                          : Skipping adding "" to URLs because it is not a directory or a JAR
2018-05-11 16:01:13.250  INFO 45 --- [           main] utoConfigurationReportLoggingInitializer : 

Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
2018-05-11 16:01:13.272 ERROR 45 --- [           main] o.s.boot.SpringApplication               : Application startup failed

java.lang.ExceptionInInitializerError: null
    at org.apache.uima.cas.admin.CASFactory.createTypeSystem(CASFactory.java:126) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.CASImpl.<init>(CASImpl.java:937) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.CASImpl.<init>(CASImpl.java:960) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.admin.CASFactory.createCAS(CASFactory.java:48) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.util.CasCreationUtils.doCreateCas(CasCreationUtils.java:611) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.util.CasCreationUtils.createCas(CasCreationUtils.java:292) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at eu.openminted.workflows.uima.executor.UIMAFitRunner.runPipeline(UIMAFitRunner.java:119) ~[classes!/:0.0.1-SNAPSHOT]
    at eu.openminted.workflows.uima.executor.UIMAFitRunner.uimaFitRun(UIMAFitRunner.java:106) ~[classes!/:0.0.1-SNAPSHOT]
    at eu.openminted.workflows.uima.executor.PipelineCommandLineRunner.run(PipelineCommandLineRunner.java:41) [classes!/:0.0.1-SNAPSHOT]
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) ~[spring-boot-1.4.2.RELEASE.jar!/:1.4.2.RELEASE]
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:784) ~[spring-boot-1.4.2.RELEASE.jar!/:1.4.2.RELEASE]
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:771) ~[spring-boot-1.4.2.RELEASE.jar!/:1.4.2.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) ~[spring-boot-1.4.2.RELEASE.jar!/:1.4.2.RELEASE]
    at eu.openminted.workflows.uima.executor.PipelineCommandLineRunner.main(PipelineCommandLineRunner.java:29) [classes!/:0.0.1-SNAPSHOT]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_161]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_161]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_161]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_161]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) [omtd-component-uima-0.0.1-SNAPSHOT-exec.jar:0.0.1-SNAPSHOT]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) [omtd-component-uima-0.0.1-SNAPSHOT-exec.jar:0.0.1-SNAPSHOT]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) [omtd-component-uima-0.0.1-SNAPSHOT-exec.jar:0.0.1-SNAPSHOT]
    at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:521) [omtd-component-uima-0.0.1-SNAPSHOT-exec.jar:0.0.1-SNAPSHOT]
Caused by: org.apache.uima.cas.CASRuntimeException: The JCas cannot be initialized.  The following errors occurred: 
JCas Class "org.apache.uima.jcas.tcas.DocumentAnnotation", loaded from "jar:file:/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar!/BOOT-INF/lib/uimaj-document-annotation-2.8.1.jar!/org/apache/uima/jcas/tcas/DocumentAnnotation.class", is missing required constructor; likely cause is wrong version (UIMA version 3 or later JCas required).

    at org.apache.uima.cas.impl.FSClassRegistry.reportErrors(FSClassRegistry.java:1284) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.FSClassRegistry.loadJCasForTSandClassLoader(FSClassRegistry.java:400) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.FSClassRegistry.getGeneratorsForClassLoader(FSClassRegistry.java:1320) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.TypeSystemImpl.getGeneratorsForClassLoader(TypeSystemImpl.java:2888) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.TypeSystemImpl.commit(TypeSystemImpl.java:1435) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.TypeSystemImpl.commit(TypeSystemImpl.java:1358) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    at org.apache.uima.cas.impl.TypeSystemImpl.<clinit>(TypeSystemImpl.java:2735) ~[uimaj-core-3.0.0.jar!/:3.0.0]
    ... 22 common frames omitted

2018-05-11 16:01:13.277  INFO 45 --- [           main] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@6e5e91e4: startup date [Fri May 11 15:59:46 UTC 2018]; root of context hierarchy
2018-05-11 16:01:13.280  INFO 45 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown

I don't have the option to create a workflow using v2. There is a button in the galaxy interface to switch to version 0.0.2, but this doesn't appear to do anything. Is it disabled? One option would be to delete v0.0.1. Is that the right way to go about this?

I am not sure where exactly are you looking. Could you please about which button are you talking about?

mattshardlow commented 6 years ago

There was a button in the Galaxy interface to switch from v0.0.1 to v0.0.2, although I didn't screenshot it and it has disappeared since I deleted the older version of the component.

It looks from the log (and from a conversation I just had with Mark on Skype) like the old version of the component was still running. We've cleared out the old version of the component and I've set a new workflow running with the correct version.

mattshardlow commented 6 years ago

It's still running, which I assume means the workflow is hanging. I don't have any info to debug with. Any thoughts on what the problem might be @gkirtzou or @galanisd ?

greenwoodma commented 6 years ago

@mattshardlow I've just had a look and I can see the new wrokflow and that it called v0.0.2 correctly. From the galaxy logs it appears that the workflow completed successfully, although strangely I can't see any output having been produced. Does it still show as running in the UI? Does your component write out the XMI files when it's finished?

gkirtzou commented 6 years ago

Here is the log from the latest run of your component. The image has finished without an error, but it seems that no input has been provided. Your component consumes XMI files, right? Have you created a workflow where you used TIKA or PDFReader to generate xmi from pdf files? You can check here for information how you can build a workflow in OMTD : https://openminted.github.io/releases/workflow-editor/1.0.0/workflow

Using default installation dir /opt/omtd-component-executor
ocoord:uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.2
19:07:37.070 [main] INFO eu.openminted.workflows.componentdependencies.DependenciesFetcherMain - ...

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.2.RELEASE)

2018-05-11 19:07:38.834  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : Starting DependenciesFetcherMain v0.0.1-SNAPSHOT on snf-774932 with PID 16 (/opt/omtd-component-executor/omtd-component-dependencies-fetcher/target/omtd-component-dependencies-fetcher-0.0.1-SNAPSHOT-exec.jar started by root in /opt/omtd-component-executor)
2018-05-11 19:07:38.842  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : No active profile set, falling back to default profiles: default
2018-05-11 19:07:39.058  INFO 16 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@22d8cfe0: startup date [Fri May 11 19:07:39 UTC 2018]; root of context hierarchy
2018-05-11 19:07:41.194  INFO 16 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-05-11 19:07:41.225  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.2
classPathListFileName:./TDMClasspathLists/classpath.uk.ac.mmu.tdmlab.journalism_TextMiningForJournalismApplication_0.0.2
uk.ac.mmu.tdmlab.journalism:TextMiningForJournalismApplication:0.0.2 

length:70
FIlTER/opt/TDMlocalRepo/org/springframework/spring-core/3.2.16.RELEASE/spring-core-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-context/3.2.16.RELEASE/spring-context-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-aop/3.2.16.RELEASE/spring-aop-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-expression/3.2.16.RELEASE/spring-expression-3.2.16.RELEASE.jar
FIlTER/opt/TDMlocalRepo/org/springframework/spring-beans/3.2.16.RELEASE/spring-beans-3.2.16.RELEASE.jar

2018-05-11 19:08:27.814  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : Started DependenciesFetcherMain in 50.042 seconds (JVM running for 52.07)
2018-05-11 19:08:27.815  INFO 16 --- [           main] e.o.w.c.DependenciesFetcherMain          : DONE!
2018-05-11 19:08:27.818  INFO 16 --- [       Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@22d8cfe0: startup date [Fri May 11 19:07:39 UTC 2018]; root of context hierarchy
2018-05-11 19:08:27.823  INFO 16 --- [       Thread-1] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown
coordinates:uk.ac.mmu.tdmlab.journalism_TextMiningForJournalismApplication_0.0.2
inDir:tmp
otDir:/srv/galaxy/database/jobs_directory/001/1414/working/out/
uimaParams:
19:08:29.149 [main] INFO eu.openminted.workflows.uima.executor.PipelineCommandLineRunner - ...

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.4.2.RELEASE)

2018-05-11 19:08:31.304  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : Starting PipelineCommandLineRunner v0.0.1-SNAPSHOT on snf-774932 with PID 45 (/opt/omtd-component-executor/omtd-component-uima/target/omtd-component-uima-0.0.1-SNAPSHOT-exec.jar started by root in /opt/omtd-component-executor/scripts/working)
2018-05-11 19:08:31.311  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : No active profile set, falling back to default profiles: default
2018-05-11 19:08:31.542  INFO 45 --- [           main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@6e5e91e4: startup date [Fri May 11 19:08:31 UTC 2018]; root of context hierarchy
2018-05-11 19:08:33.923  INFO 45 --- [           main] f.a.AutowiredAnnotationBeanPostProcessor : JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
2018-05-11 19:08:34.617  INFO 45 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
parsing...
input tmp
output /srv/galaxy/database/jobs_directory/001/1414/working/out/
className uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication
2018-05-11 19:08:34.692  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : 

Executing..
========================
className:uk.ac.mmu.tdmlab.journalism.TextMiningForJournalismApplication
inputDir:tmp
outputDir:/srv/galaxy/database/jobs_directory/001/1414/working/out/
**parameters
========================
========================

2018-05-11 19:08:35.521  WARN 45 --- [           main] org.apache.uima                          : SAXTransformerFactory didn't recognize setting attribute http://javax.xml.XMLConstants/property/accessExternalDTD
2018-05-11 19:08:35.521  WARN 45 --- [           main] org.apache.uima                          : SAXTransformerFactory didn't recognize setting attribute http://javax.xml.XMLConstants/property/accessExternalStylesheet
2018-05-11 19:08:36.326  INFO 45 --- [           main] d.t.ukp.dkpro.core.io.xmi.XmiReader      : Scanning [file:/opt/omtd-component-executor/scripts/working/tmp/]
2018-05-11 19:08:36.329  INFO 45 --- [           main] d.t.ukp.dkpro.core.io.xmi.XmiReader      : Found [0] resources to be read
2018-05-11 19:08:38.034  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator tokenize
2018-05-11 19:08:38.078  INFO 45 --- [           main] e.s.nlp.pipeline.TokenizerAnnotator      : No tokenizer type provided. Defaulting to PTBTokenizer.
2018-05-11 19:08:38.116  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator ssplit
2018-05-11 19:08:38.146  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator pos
2018-05-11 19:08:39.945  INFO 45 --- [           main] e.s.nlp.tagger.maxent.MaxentTagger       : Loading POS tagger from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [1.7 sec].
2018-05-11 19:08:39.947  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator lemma
2018-05-11 19:08:39.951  INFO 45 --- [           main] e.stanford.nlp.pipeline.StanfordCoreNLP  : Adding annotator ner
2018-05-11 19:08:45.445  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.all.3class.distsim.crf.ser.gz ... done [3.4 sec].
2018-05-11 19:08:50.700  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.muc.7class.distsim.crf.ser.gz ... done [5.3 sec].
2018-05-11 19:08:51.767  INFO 45 --- [           main] e.s.nlp.ie.AbstractSequenceClassifier    : Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... done [1.1 sec].
2018-05-11 19:08:51.780  INFO 45 --- [           main] edu.stanford.nlp.time.JollyDayHolidays   : Initializing JollyDayHoliday for SUTime from classpath edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.
2018-05-11 19:08:52.455  INFO 45 --- [           main] e.s.n.time.TimeExpressionExtractorImpl   : Using following SUTime rules: edu/stanford/nlp/models/sutime/defs.sutime.txt,edu/stanford/nlp/models/sutime/english.sutime.txt,edu/stanford/nlp/models/sutime/english.holidays.sutime.txt
2018-05-11 19:09:04.099  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 580641 unique entries out of 581790 from edu/stanford/nlp/models/kbp/regexner_caseless.tab, 0 TokensRegex patterns.
2018-05-11 19:09:04.151  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 4857 unique entries out of 4868 from edu/stanford/nlp/models/kbp/regexner_cased.tab, 0 TokensRegex patterns.
2018-05-11 19:09:04.152  INFO 45 --- [           main] e.s.n.pipeline.TokensRegexNERAnnotator   : TokensRegexNERAnnotator ner.fine.regexner: Read 585498 unique entries from 2 files
2018-05-11 19:09:32.837  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : Started PipelineCommandLineRunner in 63.143 seconds (JVM running for 64.893)
2018-05-11 19:09:32.838  INFO 45 --- [           main] e.o.w.u.e.PipelineCommandLineRunner      : DONE!
2018-05-11 19:09:32.842  INFO 45 --- [       Thread-1] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@6e5e91e4: startup date [Fri May 11 19:08:31 UTC 2018]; root of context hierarchy
2018-05-11 19:09:32.848  INFO 45 --- [       Thread-1] o.s.j.e.a.AnnotationMBeanExporter        : Unregistering JMX-exposed beans on shutdown

mattshardlow commented 6 years ago

See image of my workflow attached. I'm very surprised that it didn't find any input.

My component is a standard UIMA Analysis Engine, so the docker wrapping process should configure it correctly right?

It's not clear to me why the application didn't show that it had finished? It still says that it is running in the 'my operations' section. Is this a known bug?

greenwoodma commented 6 years ago

It's not clear to me why the application didn't show that it had finished? It still says that it is running in the 'my operations' section. Is this a known bug?

Not a known bug but I can guess what's happened. Essentially the workflow service waits for the final job to be finished and then waits for the output to appear. Given there is no output I'm guessing it's still sat waiting. I'll have a look and see if I can find a solution (the problem of spotting a workflow having finished is awkward via the API and hence we were relying in part on outputs being produced).

mattshardlow commented 6 years ago

Thanks Mark.

I've tried to run the application with a few other corpora (a brand new testing corpus by penny, Named Entity Recognition Corpus for Social Science Publications) - but in both cases I got an error, the first one saying that the corpus could not be found and the second one saying that the corpus was empty. I'll keep looking for a solution.

mattshardlow commented 6 years ago

see:

mattshardlow commented 6 years ago

I changed the TIKA pattern from */.pdf to */.* as it says in the documentation at: https://openminted.github.io/releases/workflow-editor/1.0.0/workflow

This is now running - however I have no way to tell whether it is running and has completed with no output.

On a separate note, I don't seem to be able to edit the parameters of components when I return to a workflow. See attached image, with Tika highlighted, but no details on the right hand side.

greenwoodma commented 6 years ago

On a separate note, I don't seem to be able to edit the parameters of components when I return to a workflow. See attached image, with Tika highlighted, but no details on the right hand side.

that's a known bug in Galaxy which we hope will be fixed soon. See issue #10 for all the gory details. The current workaround is to delete and re-add the component you want to edit.

mattshardlow commented 6 years ago

Thanks Mark. Good to know.

greenwoodma commented 6 years ago

I changed the TIKA pattern from */.pdf to /. as it says in the documentation at: https://openminted.github.io/releases/workflow-editor/1.0.0/workflow

This is now running - however I have no way to tell whether it is running and has completed with no output.

Not sure if it's an error in copy-paste or not but to select PDF files you need the pattern **/*.pdf your example misses the * after the /. In the same way to select all files I would assume you need **/* or more likely **/*.*.

I've had a look at the logs and on this occasion it failed when running the omtdImporter suggesting something wrong with the corpus, although I can't see what exactly. What's weird is I would expect it to report the failure in the UI. Does it still show as running?

mattshardlow commented 6 years ago

Yeah, that's an issue with the markdown swallowing some of the characters. It should read

"*\*\/*.*"

I originally had the */.pdf, but changed it following the docs at https://openminted.github.io/releases/workflow-editor/1.0.0/workflow, which say:

Caution! For both converters you need to configure the pattern parameter (displayed in the right column): for the Tika Multi-Format Reader, set the pattern to "*\*\/*.*" and for PDFReader, set the pattern to "*/.pdf".

I'll try a few other patterns to see if that's the issue

greenwoodma commented 6 years ago

Hmm, that Tika pattern looks like a typo. It would mean that the directory had to end with a literal * which makes no sense.

@galanisd, I think you wrote the Tika converter, so should the docs for that read **/*.* instead of *\*/*.*

reckart commented 6 years ago

Now that you mention it, I think @pennyl67 asked me to check that pattern before converting the adoc into HTML and to do a proper escaping... so I guess it is my fault.

greenwoodma commented 6 years ago

so I guess it is my fault.

I just know that for once it't not my fault :rofl:

reckart commented 6 years ago

Should be fixed by this commit (https://github.com/openminted/openminted.github.io/commit/d127bda28997ad3b5b9b3749c5f84db8da9c7604) and that should be live on the website in a jiffy.

reckart commented 6 years ago

@greenwoodma btw, the Tika converter is probably this one: https://dkpro.github.io/dkpro-core/releases/1.9.1/docs/format-reference.html#format-Tika123

greenwoodma commented 6 years ago

@reckart oh yes, I forgot we were using that, I just remember @galanisd talking about the wrapper for Galaxy.

mattshardlow commented 6 years ago

So I currently have a workflow set up with omtdImporter -> PDFReader and nothing after that. So, as far as I uderstand it, the output from the pdf reader (e.g. the serialised cas xmi) should become the output of the workflow. However, having run that workflow a few times, I am getting the same issue as when I had my component in there (e.g., nothing happening). Any thoughts on this? If the very basic workflow doesn't complete, then It's impossible to debug my own component.

greenwoodma commented 6 years ago

@mattshardlow can you give us the OMTD ID (that long alphanumeric string from the bottom of the landing page) for the PDFReader workflow. There is clearly something odd going on that we need to solve

mattshardlow commented 6 years ago

@greenwoodma it's: 0931732053353301-7bdad7b1-673f-4007-a02b-e476fa01aa1a

My thought is that it might be an issue with the corpus. I have been trying to use the Chebi 2 pdf corpus - test 2 (a24f49a0-ade4-4df7-ab03-8086f93a7235). But this one always seems to fail. I've also tried a few other corpora, but with varying errors (see my post above).

Thanks for having a look at this. I've had a look through the other issues, am I right in thinking nobody else has tried to use the pdf reader as part of their application? If somebody else has, I could take a look at their workflow / try using their corpus with the same input configuration and then plug my component at the end.

greenwoodma commented 6 years ago

I've had a look through the other issues, am I right in thinking nobody else has tried to use the pdf reader as part of their application?

I think almost every workflow that's been built and tested so far uses the PDFReader component. As you say I think the problem is going to be with the corpus. Why not upload a couple of PDF files that are relevant to your use case and create your own test corpus? Just remember when you upload a zip file the documents need to be in the fulltext folder and not in the root of the zip file.

mattshardlow commented 6 years ago

Why not upload a couple of PDF files that are relevant to your use case and create your own test corpus?

I can give it a go. My use case is fairly flexible (as a journalist might be interested in any scientific domain). So I'll try a few of the other domain specific corpora first.

mattshardlow commented 6 years ago

I've uploaded a couple of pdfs and I'm now running both workflows on this new corpus.

I also tried to use the corpus builder. But the corpora seem to get stuck in the 'under construction' phase. (I tried one from OpenAire and one from CORE). I got 'success' emails for both with link(s): https://test.openminted.eu/landingPage/corpus/b42a3789-933a-4813-b0e9-89bfa75a827c

But the 'my corpora' page shows they are still 'under construction'.

Is this a known issue with the corpus builder? It's fairly important to the app that I wanted to develop using omtd.

galanisd commented 6 years ago

Hi Matt,

I am trying to create an app for your workflow (omtdImporter -> PDFReader -> Text Mining For Journalism Application (V2)) so that I can test, however, I am getting a "Error saving application (undefined)" message. It seems that is an AAI issue that causes various problems in the Registry but not sure that it is one of them. @courado @antleb are looking into the problem. When everything is back to normal I will create an app and run some tests to see what is happening.
The workflow in Galaxy executor that corresponds to the ID that you sent ( 0931732053353301-7bdad7b1-673f-4007-a02b-e476fa01aa1a ) is the following. Galaxy-Workflow-0931732053353301-7bdad7b1-673f-4007-a02b-e476fa01aa1a.ga --> {"uuid": "8edafe15-ca38-4c35-972c-f7958f79de1b", "tags": [], "format-version": "0.1", "name": "0931732053353301-7bdad7b1-673f-4007-a02b-e476fa01aa1a", "steps": {"0": {"tool_id": "omtdImporter", "tool_version": "1.0.0", "outputs": [{"type": "input", "name": "output"}], "workflow_outputs": [], "input_connections": {}, "tool_state": "{\"omtdStoreCorpusID\": \"\\\"\\\"\", \"__rerun_remap_job_id__\": null, \"__page__\": null}", "id": 0, "uuid": "414fb090-33d8-460c-8254-8ed2c5bacc5c", "errors": null, "name": "omtdImporter", "post_job_actions": {}, "label": null, "inputs": [], "position": {"top": 174, "left": 200}, "annotation": "", "content_id": "omtdImporter", "type": "tool"}, "1": {"tool_id": "mvn:org.dkpro.core:dkpro-core-io-tika-asl:1.9.1-SNAPSHOT#org.dkpro.core.io.tika.TikaReader", "tool_version": "1.9.1-SNAPSHOT", "outputs": [{"type": "input", "name": "output"}], "workflow_outputs": [], "input_connections": {"component_InputFiles": {"output_name": "output", "id": 0}}, "tool_state": "{\"__page__\": null, \"language\": \"\\\"\\\"\", \"__rerun_remap_job_id__\": null, \"patterns\": \"\\\"**/*.pdf\\\"\", \"useDefaultExcludes\": \"\\\"true\\\"\", \"component_InputFiles\": \"{\\\"__class__\\\": \\\"RuntimeValue\\\"}\", \"logFreq\": \"\\\"1\\\"\", \"sourceLocation\": \"\\\"\\\"\", \"includeHidden\": \"\\\"false\\\"\"}", "id": 1, "uuid": "bb2a3c4b-d4dc-4a6a-932f-7b134c98b233", "errors": null, "name": "Tika Multi-Format Reader", "post_job_actions": {}, "label": null, "inputs": [{"name": "component_InputFiles", "description": "runtime parameter for tool Tika Multi-Format Reader"}], "position": {"top": 174, "left": 364.5}, "annotation": "", "content_id": "mvn:org.dkpro.core:dkpro-core-io-tika-asl:1.9.1-SNAPSHOT#org.dkpro.core.io.tika.TikaReader", "type": "tool"}, "2": {"tool_id": "mvn:de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.9.0#de.tudarmstadt.ukp.dkpro.core.opennlp.OpenNlpSegmenter", "tool_version": "1.9.0", "outputs": [{"type": "input", "name": "output"}], "workflow_outputs": [], "input_connections": {"component_InputFiles": {"output_name": "output", "id": 1}}, "tool_state": "{\"__page__\": null, \"writeToken\": \"\\\"true\\\"\", \"language\": \"\\\"\\\"\", \"segmentationModelLocation\": \"\\\"\\\"\", \"strictZoning\": \"\\\"false\\\"\", \"zoneTypes\": \"\\\"de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Div\\\"\", \"writeSentence\": \"\\\"true\\\"\", \"component_InputFiles\": \"null\", \"modelVariant\": \"\\\"\\\"\", \"tokenizationModelLocation\": \"\\\"\\\"\", \"__rerun_remap_job_id__\": null, \"writeForm\": \"\\\"true\\\"\"}", "id": 2, "uuid": "0d2d347a-af26-47d1-a4c9-624a833846ed", "errors": null, "name": "OpenNLP Segmenter", "post_job_actions": {}, "label": null, "inputs": [], "position": {"top": 202, "left": 596}, "annotation": "", "content_id": "mvn:de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core.opennlp-asl:1.9.0#de.tudarmstadt.ukp.dkpro.core.opennlp.OpenNlpSegmenter", "type": "tool"}}, "annotation": "", "a_galaxy_workflow": "true"}

I thought that you have created a workflow with omtdImporter and PDFReader not OpenNLPSegmenter. Maybe I missed something.

mattshardlow commented 6 years ago

@galanisd Yes, I originally had the Segmenter in as well, but I then deleted it from the workflow to simplify it. Looking at it now, it appears that change wasn't reflected when I saved the workflow previously. I've tried deleting it again and saving it, but the openNLPSegmenter still reappears. Is this a known issue? There is no message to indicate that the save failed.

Thanks for having a look at the debug. That's strange with the AAAI error - could this be related to the workflow not saving? I've had a look at some other workflows and it appears that other changes haven't been saved either.

gkirtzou commented 6 years ago

I've had a look at the logs and on this occasion it failed when running the omtdImporter suggesting something wrong with the corpus, although I can't see what exactly. What's weird is I would expect it to report the failure in the UI. Does it still show as running?

I have a suspicion (I didn't have time to verify it) that when there is a workflow with multiple steps (eg 4 step) and an error occurs in one of the inner steps (eg the 2nd), then the workflow engine is not able to understand that something went wrong and propagate the error to the registry correctly.

In general when the error occurs in the final step, the UI shows the "An error occured" message. In the other case, I think it is under the impression that it is still running.

greenwoodma commented 6 years ago

I agree @gkirtzou it appears that way but.... I specifically updated the workflow service a couple of weeks ago to handle this case. It now scans each step, not just the last, for the error state and should report the error in all cases. I'm wondering if the workflow service on test hasn't been updated properly. @antleb do you know when the workflow service was last updated on test and if it's using the latest version or not?

mattshardlow commented 6 years ago

@galanisd have you been able to test the component? Is there anything more I can do from my side? I don;t want to run more tests if they might interfere with your results? Let me know.

galanisd commented 6 years ago

Currently there are two issues in the Registry and it is not possible to test the app that I created. Waiting fixes for both of them.

However, locally in my laptop I tested your component using the UIMA executor script that is also used in OMTD platform. As input I used two random XMI files. It finished successfully so I expect the same to happen and in the platform when the issues that I mentioned are resolved. The output contains some "journalism" annotations; see below some examples.

<journalism:Where xmi:id="1679" sofa="720" begin="1448" end="1456"/><journalism:Where xmi:id="1683" sofa="720" begin="5666" end="5674"/><journalism:Where xmi:id="1687" sofa="720" begin="5749" end="5757"/><journalism:Where xmi:id="1691" sofa="720" begin="5805" end="5815"/><journalism:Where xmi:id="1695" sofa="720" begin="10467" end="10473"/><journalism:Where xmi:id="1699" sofa="720" begin="11800" end="11806"/><journalism:Where xmi:id="1703" sofa="720" begin="11902" end="11910"/><journalism:Where xmi:id="1707" sofa="720" begin="13692" end="13700"/><journalism:Where xmi:id="1711" sofa="720" begin="13936" end="13942"/><journalism:Where xmi:id="1715" sofa="720" begin="13943" end="13956"/><journalism:Where xmi:id="1719" sofa="720" begin="14351" end="14359"/><journalism:Where xmi:id="1723" sofa="720" begin="14865" end="14873"/><journalism:Where xmi:id="1727" sofa="720" begin="16122" end="16130"/><journalism:Where xmi:id="1731" sofa="720" begin="16159" end="16166"/><journalism:Where xmi:id="1735" sofa="720" begin="16174" end="16184"/><journalism:Where xmi:id="1739" sofa="720" begin="20922" end="20928"/><journalism:Where xmi:id="1743" sofa="720" begin="21345" end="21353"/><journalism:Where xmi:id="1747" sofa="720" begin="21356" end="21364"/><journalism:Where xmi:id="1751" sofa="720" begin="21486" end="21494"/><journalism:Where xmi:id="1755" sofa="720" begin="22256" end="22264"/><journalism:Where xmi:id="1759" sofa="720" begin="22297" end="22303"/><journalism:Where xmi:id="1763" sofa="720" begin="22731" end="22739"/><journalism:Where xmi:id="1767" sofa="720" begin="22743" end="22752"/><journalism:Where xmi:id="1771" sofa="720" begin="23019" end="23027"/><journalism:Where xmi:id="1775" sofa="720" begin="23684" end="23692"/><journalism:Where xmi:id="1779" sofa="720" begin="23772" end="23794"/><journalism:Where xmi:id="1783" sofa="720" begin="23861" end="23868"/><journalism:Where xmi:id="1787" sofa="720" begin="24900" end="24910"/><journalism:Where xmi:id="1791" sofa="720" begin="26625" end="26630"/><journalism:Where xmi:id="1795" sofa="720" begin="26642" end="26650"/><journalism:Where xmi:id="1799" sofa="720" begin="26687" end="26693"/><journalism:Where xmi:id="1803" sofa="720" begin="26713" end="26721"/><journalism:Where xmi:id="1807" sofa="720" begin="26815" end="26824"/><journalism:Where xmi:id="1811" sofa="720" begin="26894" end="26902"/><journalism:Where xmi:id="1815" sofa="720" begin="27019" end="27027"/><journalism:Where xmi:id="1819" sofa="720" begin="27120" end="27125"/><journalism:Where xmi:id="1823" sofa="720" begin="27147" end="27155"/><journalism:Where xmi:id="1827" sofa="720" begin="27311" end="27319"/><journalism:Where xmi:id="1831" sofa="720" begin="27510" end="27518"/><journalism:Where xmi:id="1835" sofa="720" begin="27531" end="27540"/><journalism:Where xmi:id="1839" sofa="720" begin="27780" end="27786"/><journalism:Where xmi:id="1843" sofa="720" begin="27799" end="27816"/>

So I think that your component (uk.ac.mmu.tdmlab.journalism, TextMiningForJournalismApplication, 0.0.2) is compatible with OMTD software.

mattshardlow commented 6 years ago

Thanks Dimitris, that's excellent news. Yes, those look like the right type of output annotations in the xmi. Is there any time scale on the fixes? Will they be ready before the launch?

My understanding is that the annotation viewer is currently not working on the platform? Is this likely to be resolved before the launch? If not, I will need to think of another way to show the results of my workflow.

mattshardlow commented 6 years ago

@galanisd any update on fixing the issues in the registry? I'd like to have a go with the component soon so as I can make the video for the presentation. Do you think I'll be able to do this by the start of next week?

galanisd commented 6 years ago

I have just run a test @ test.openminted.eu. Selected a random corpus + the journalism app that I have created.

matt

The workflow is this.

matt

The workflow has finished successfully. I have checked Galaxy. Output attached. component_output.zip

There is still some issue in Registry and the status of the execution remains "RUNNING" for ever. @antleb @courado are working on it. The good news are that your component is compatible with OMTD software (as I expected). @gkirtzou @mattshardlow. The metadata have been checked?

mattshardlow commented 6 years ago

Thanks for testing this Dimitris, that's good to hear. Does that mean that the output won't be available in the registry whilst the issue with the execution not flagging as finished persists.

My understanding is that this occurs because Galaxy isn't picking up any output from the workflow. If I put an extra component on the end that outputs a file, then would this circumvent the problem for now?

I have a simple component which summarises the annotations in the CAS, that I could upload to the platform for this purpose.

see: https://github.com/MMU-TDMLab/UIMA/tree/master/AnnotationSummariser (not much documentation unfortunately)

galanisd commented 6 years ago

My understanding is that this occurs because Galaxy isn't picking up any output from the workflow. If I put an extra component on the end that outputs a file, then would this circumvent the problem for now?

No the issue as I said is in Registry.

One solution is to register your component @ services.openminted.eu; no issues there when a workflow finishes as far as I know. Please keep the component private, download the metadata and send them to us. Katerina (@gkirtzou ) will check them and if required she will suggest some modifications. Then you will be able to make the component public create a workflow and run with some corpus. This way you will be able to create the video that you need for the presentation.

gkirtzou commented 6 years ago

@mattshardlow Could you proceed registering your component to services? Please register it as private, and send me the metadata, so I could provide you feedback.

mattshardlow commented 6 years ago

@gkirtzou Have uploaded. See metadata attached. I checked for consistency with the component on test, so it should be ok. I'll put together an application too and test + send you the metadata.

89c0fd9d-7f2e-4fe9-9565-39e6de7b05fa.xml.txt

mattshardlow commented 6 years ago

metadata for the test corpus: 10358254-c42e-4850-8918-5c0b16a7cb22.xml.txt

and for the application: ec177220-f05c-4131-987a-f83003b4315e.xml.txt

I've set the application running (configured as Dimitris showed above). Just waiting for it to finish.

mattshardlow commented 6 years ago

It appeared that the workflow was not stopping as in test. I created a new version with the same config, apart from 2 modifications: 1) I clicked the star next to the output which indicates that the output is a workflow output. 2) I set it to notify me when the application finished via email. I just recieved an email, and indeed the application appears to have finished.

However, although I can see the pdfs, they do not appear to have any annotations on top of them. Is it possible to view the output CAS via the Annotation Viewer that Jacob developed?

openminted / Open-Call-Discussions

MMU - Hackathon #37