search

riedlma / topictiling

TopicTiling is a text segmentation method that is based on LDA
GNU General Public License v3.0
40 stars 10 forks source link

org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl callAnalysisComponentProcess(407) #6

Open feider opened 3 years ago

feider commented 3 years ago

Hi,

my issue may be related to issue #4

I call the tiling script with sh topictiling.sh -tmd ../topicmodel -s -tmn model-final -fp "Wigalois.txt" -fd ../../data/pdf/ascii/ -out results -d

The LDA model (generated with jgibblda) and the file seem to be read correctly and the file is printed out using the -d option. I use -s for simple segmentation, but also tried adding a . at the end of every line instead. The text is in Middle High German, but converted to ASCII characters. Here is an example:

wer hat mich guoter uf getan
si ez iemen der mich kan
beidiu lesen und versten
der sol genade an mir begen
ob iht wandels an mir si
daz er mich doch laze vri
valscher rede daz eret in
ich weiz wol daz ich niene bin
geliutert und gerihtet
noch so wol getihtet

The full error output is here:

pr 17, 2021 3:17:44 PM org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl callAnalysisComponentProcess(407)
SEVERE: Exception occurred
org.apache.uima.analysis_engine.AnalysisEngineProcessException: Annotator processing failed.  
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:391)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.processAndOutputNewCASes(PrimitiveAnalysisEngine_impl.java:296)
    at org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase.process(AnalysisEngineImplBase.java:267)
    at org.uimafit.pipeline.SimplePipeline.runPipeline(SimplePipeline.java:223)
    at org.uimafit.pipeline.SimplePipeline.runPipeline(SimplePipeline.java:143)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.RunTopicTilingOnFile.<init>(RunTopicTilingOnFile.java:133)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.RunTopicTilingOnFile.main(RunTopicTilingOnFile.java:94)
Caused by: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0
    at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
    at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
    at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
    at java.base/java.util.Objects.checkIndex(Objects.java:359)
    at java.base/java.util.ArrayList.get(ArrayList.java:427)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.annotator.TopicTilingSegmenterAnnotator.annotateSegments(TopicTilingSegmenterAnnotator.java:231)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.annotator.TopicTilingSegmenterAnnotator.process(TopicTilingSegmenterAnnotator.java:142)
    at org.apache.uima.analysis_component.JCasAnnotator_ImplBase.process(JCasAnnotator_ImplBase.java:48)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:375)
    ... 6 more

Exception in thread "main" org.apache.uima.analysis_engine.AnalysisEngineProcessException: Annotator processing failed.
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:391)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.processAndOutputNewCASes(PrimitiveAnalysisEngine_impl.java:296)
    at org.apache.uima.analysis_engine.impl.AnalysisEngineImplBase.process(AnalysisEngineImplBase.java:267)
    at org.uimafit.pipeline.SimplePipeline.runPipeline(SimplePipeline.java:223)
    at org.uimafit.pipeline.SimplePipeline.runPipeline(SimplePipeline.java:143)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.RunTopicTilingOnFile.<init>(RunTopicTilingOnFile.java:133)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.RunTopicTilingOnFile.main(RunTopicTilingOnFile.java:94)
Caused by: java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 0
    at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
    at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
    at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
    at java.base/java.util.Objects.checkIndex(Objects.java:359)
    at java.base/java.util.ArrayList.get(ArrayList.java:427)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.annotator.TopicTilingSegmenterAnnotator.annotateSegments(TopicTilingSegmenterAnnotator.java:231)
    at de.tudarmstadt.langtech.semantics.segmentation.segmenter.annotator.TopicTilingSegmenterAnnotator.process(TopicTilingSegmenterAnnotator.java:142)
    at org.apache.uima.analysis_component.JCasAnnotator_ImplBase.process(JCasAnnotator_ImplBase.java:48)
    at org.apache.uima.analysis_engine.impl.PrimitiveAnalysisEngine_impl.callAnalysisComponentProcess(PrimitiveAnalysisEngine_impl.java:375)
    ... 6 more

I tried openjdk-15, openjdk-7 and the current oracle jre. Is there anything I'm doing wrong or anything different that I can try?

Kind regards

patsab commented 3 years ago

I think the each line is interpreted as a new document. With adding a "." to the end of the line, there are still no multiple sentences per document. Can you try to add all the different lines into a single line, then add a "." between them ? So the line would look like: wer hat mich guoter uf getan . si ez iemen der mich kan . beidiu lesen und versten . ...

feider commented 3 years ago

Thank you for the suggestion, but that did not help. This way it just reordered the sentences/lines, similar to each verse on a single line with the -s option.