search

clulab / eidos

Machine reading system for World Modelers
Apache License 2.0
36 stars 24 forks source link

What if processors was updated #1132

Closed kwalcock closed 1 year ago

kwalcock commented 2 years ago

This comment was moved from https://github.com/clulab/processors/pull/643

The problem above was solved with

    "org.apache.lucene"           % "lucene-backward-codecs"   % luceneVer,

After that, the tests run. For my records, these below are then failing. I suspect that it comes from changing of tags, like:

TO -> IN nmod -> obl nmod_to -> obl_to dobj -> obj

I don't know whether it's worth updating any rules or hard-coded tags. Eidos doesn't necessarily need to use this processors update.

[error] Failed tests:
[error]         org.clulab.wm.eidos.text.english.raps.TestRaps
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc5
[error]         org.clulab.wm.eidos.serialization.jsonld.TestJLDSerializer
[error]         org.clulab.wm.eidos.text.english.raps.TestRaps1
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc8
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP1
[error]         org.clulab.wm.eidos.text.english.cag.TestExtraText
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP0
[error]         org.clulab.wm.eidos.text.englishGrounding.TestSpecificGroundings
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc2
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP4
[error]         org.clulab.wm.eidos.system.TestHedging
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc3
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc6
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP3
[error]         org.clulab.wm.eidos.system.TestNegation
[error]         org.clulab.wm.eidos.rule.TestJointAdjectives
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc1
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc4
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP6
[error]         org.clulab.wm.eidos.system.TestEidosMention
[error]         org.clulab.wm.eidos.text.english.cag.TestCagP2
[error]         org.clulab.wm.eidos.utils.TestMentionUtils
[error]         org.clulab.wm.eidos.system.TestEidosActions
[error]         org.clulab.wm.eidos.system.TestFiltering
[error]         org.clulab.wm.eidos.text.english.eval6.TestDoc7
kwalcock commented 2 years ago

Example fails for one test file, TestDoc1 (org.clulab.wm.eidos.text.english.eval6.TestDoc1), are

[info] TestDoc1 Paragraph 2
[info] - should have correct edge 1 *** FAILED ***

[info] TestDoc1 Paragraph 4
[info] - should have correct edge 2 *** FAILED ***

[info] TestDoc1 Paragraph 5
[info] - should have correct edge 1 *** FAILED ***
[info] - should have correct edge 6 *** FAILED ***
[info] - should have correct edge 7 *** FAILED ***

Unfortunately, there is no record of which rule needed to match in order to get the missing edge. One could debug with the old version, find the Mention that makes the same test pass, and read the rule out of that. It might be faster and more thorough to look through all the rules for things that would be affected by changes in processors/Stanford CoreNLP and take care of them all. Passing the tests would be a double check. That would at least take care of name changes. Changed behavior would be more difficult to spot.