clulab / eidos

Machine reading system for World Modelers
Apache License 2.0
36 stars 24 forks source link

TestSentenceClassifier seems to test texts without sentences #1094

Open kwalcock opened 2 years ago

kwalcock commented 2 years ago

At https://github.com/clulab/eidos/blob/6d1912299fb1aeabdac5cfc7f749a1acdcc2039e/src/test/scala/org/clulab/wm/eidos/document/TestSentenceClassifier.scala#L138-L145, the test assigns 0 to the prediction and takes whatever label there is to make a calculation even when there is no sentence. I don't think this should happen, Both numbers should be thrown out as if it never happened. In the four cases, there are run-on sentences from tables or captions that we are skipping.