kordjamshidi
commented
7 years ago @Rahgooy : any guess about what is this error? @bhargav : is it relevant for the SRL in the pipeline and TextAnnotation?
Closed PattyBG closed 7 years ago
kordjamshidi
commented
7 years ago @Rahgooy : any guess about what is this error? @bhargav : is it relevant for the SRL in the pipeline and TextAnnotation?
bhargav
commented
7 years ago The failure happens in the SRL annotator in the pipeline. But I couldn't find the exact reason for the error. Does this happen on all test sentences or any specific sentence?
PattyBG
commented
7 years ago In "saul-example", I already had to create a folder "models/sprl/2012/Relation" to make the training work ... Whithout the creation of this folder, it didn't work. Are you sure there is everything needed for Spatial Role Labeling on GitHub ?
Rahgooy
commented
7 years ago Please train/test without SRL feature(actually the results reported are without this feature):
val JF2_12 = property(relations) {
x: SpRelation =>""
}TextAnnotationFactory.disableSettings(settings, USE_SRL_NOM, USE_NER_ONTONOTES, USE_SRL_VERB)@PattyBG Please let us know if the problem is just from SRL feature or not.
PattyBG
commented
7 years ago I manage to run the testing part on the 2013 dataset provided here : http://www.cs.tulane.edu/~pkordjam/SpRL.htm. (to switch from training to testing, i put public static final Property IS_TRAINING = new Property("isTraining", Configurator.FALSE, is it a good way to procede?)
However, I would like to test with my own sentences. How should I do that ? I tried to change the SpRL_SemEval2013_test.xml file by :
<SpRL>
<TEXT>A man with sunglasses sits on a chair .</TEXT>
<TAGS></TAGS>
</SpRL>But it doesn't work.
Thanks for your time :)
Rahgooy
commented
7 years ago Your xml file is right, but you need to print the results instead of evaluation(classifier.test()). You can use something like this here :
//classifier.test()
SpRLDataModel.relations().filter(r => classifier(r).equalsIgnoreCase("Gold"))
.foreach(r => {
println(r.getSentence.getText)
println(s"tr:${r.getTrajector.getText}")
println(s"sp:${r.getSpatialIndicator.getText}")
if (r.landmarkIsDefined())
println(s"Lm:${r.getLandmark.getText}")
else
println("lm: None")
})Thanks for your quick answer :) I changed the code and it works fine with the "SpRL_SemEval2013_test.xml" file provided here : http://www.cs.tulane.edu/~pkordjam/SpRL.htm. However I still don't understand how to get (Trajector, Indicator, etc..) for my own sentences I want to annotate.
I am a bit lost ... I don't understand what is this "SpRL_SemEval2013_gold.xml" file ... In it, we have the sentences of your "SpRL_SemEval2013_test.xml" and the relations (Trajector, Indicator, etc..) at the end. Why ? Should I change the "SpRL_SemEval2013_gold.xml" file in order to test with my own sentences ?? Or just the SpRL_SemEval2013_test.xml" ?
Rahgooy
commented
7 years ago The test SpRL_SemEval2013_test.xml includes sentences that are not annotated, so in the 2013 shared task those plain sentences were given to the participants such that their systems ran on that and generated the annotations. The generated annotations then compared to the SpRL_SemEval2013_gold.xml which contains ground truth annotations for the same sentences. By comparing the gold against the predictions of task participants we could report the precision/recall/f1 of their models. Does this make sense?
If you want to annotate your own sentences, you can put them in a file similar to the test xml file and change the path in the configuration here. Note that the xml reader uses a path, so put your file in a folder and point to that folder. you can see the output annotations in the console using the code I posted before. But if you want xml output you should write some code...
Hi :) I am trying to use the Spatial Role Labeling tool. I managed to do the Training Step, but I have the following error with the Testing step:
The error seems to come from
What does that mean?
Thanks a lot for your help