Information structure affixes work unfinished

olzama commented 3 years ago

After Mike write rtest, we noticed that some of the information structure tests were failing all along and we were not noticing (because the old system was not comparing ICONS).

These 8 tests are all information structure affixes tests. I investigated and was unable to locate any code that does this work (see here: https://delphinqa.ling.washington.edu/t/lost-icons-in-some-of-the-infostructure-tests/543/6?u=olzama).

I created new lexical rules for a couple of tests but I need to stop now as this isn't actually my job for the quarter. The remaining tests are:

infostr-foc-affix-after-noun infostr-foc-affix-obj-verb infostr-foc-affix-subj-obj infostr-foc-affix-subj-obj-hier infostr-foc-affix-subj-obj-verb infostr-foc-affix-subj-verb

They all require some analytical work figuring out what the lexical rules should look like and possibly also some work on the head-subject and head-complement rules.

I will now merge in the two new lexical rules, tag the revision, and then remove the remaining failing 6 tests (listed above). Once someone is assigned with finishing this work, the tests will need to be restored using the tag.

olzama commented 3 years ago

Whoever picks up this work can find the above information structure tests here: https://github.com/delph-in/matrix/releases/tag/Last-version-with-6-failing-infostr-affix-tests

I will delete them from the trunk at this point.

goodmami commented 3 years ago

Why not just delete the corresponding lines from regression-test-index so the tests are skipped (for now)?

olzama commented 3 years ago

Oh, that's a good idea, maybe? I am trying to think whether this will be more or less confusing for someone coming in, say, a few years from now. Hmm.

goodmami commented 3 years ago

This is what we do for the huge Sahaptin test. Also later we can change rtest.py so it shows that some tests were skipped, which could help future Matrix developers.

olzama commented 3 years ago

Like in terms of figuring out what is going on. I think yes, you are right; having an small text file which can simply be appended to the existing index might be the easiest. Perhaps we should even have a visible file in the trunk which lists tests like that.

This is what we do for the huge Sahaptin test.

And it did cause some confusion, as I remember :). But that's OK maybe.

Also later we can change rtest.py so it shows that some tests were skipped, which could help future Matrix developers.

That is a great idea.

emilymbender commented 3 years ago

I like the idea of keeping skipped regression tests in the repo and having the script just note which ones were skipped. Seems more likely they will one day get fixed :)

olzama commented 3 years ago

Some of the lexical rules can actually be found in matrix.tdl (but I still cannot find any code that would add those rules anywhere):

;;; icons-realted lexrules
no-icons-lexrule := lex-rule & [ C-CONT.ICONS.LIST < > ].

add-icons-rule := lex-rule &
  [ SYNSEM.LOCAL.CONT.HOOK [ INDEX #index,
                             ICONS-KEY #icons ],
    C-CONT.ICONS.LIST < info-str & #icons & [ IARG2 #index ] > ].

add-icons-subj-rule := lex-rule &
  [ SYNSEM.LOCAL.CAT.VAL.SUBJ.FIRST.LOCAL.CONT.HOOK [ INDEX #index,
                                                      ICONS-KEY #icons ],
    C-CONT.ICONS.LIST < info-str & #icons & [ IARG2 #index ] > ].

add-icons-comp-rule := lex-rule &
  [ SYNSEM.LOCAL.CAT.VAL.COMPS.FIRST.LOCAL.CONT.HOOK [ INDEX #index,
                                                       ICONS-KEY #icons ],
    C-CONT.ICONS.LIST < info-str & #icons & [ IARG2 #index ] > ].

add-icons-subj-verb-rule := lex-rule &
  [ SYNSEM.LOCAL [ CONT.HOOK [ INDEX #index1,
                   ICONS-KEY #icons1 ],
           CAT.VAL.SUBJ.FIRST.LOCAL.CONT.HOOK [ INDEX #index2,
                            ICONS-KEY #icons2 ] ],
    C-CONT.ICONS.LIST < info-str & #icons1 & [ IARG2 #index1 ],
            info-str & #icons2 & [ IARG2 #index2 ] > ].

add-icons-comp-verb-rule := lex-rule &
  [ SYNSEM.LOCAL [ CONT.HOOK [ INDEX #index1,
                   ICONS-KEY #icons1 ],
           CAT.VAL.COMPS.FIRST.LOCAL.CONT.HOOK [ INDEX #index2,
                            ICONS-KEY #icons2 ] ],
    C-CONT.ICONS.LIST < info-str & #icons1 & [ IARG2 #index1 ],
            info-str & #icons2 & [ IARG2 #index2 ] > ].

add-icons-subj-comp-rule := lex-rule &
  [ SYNSEM.LOCAL.CAT.VAL [ SUBJ.FIRST.LOCAL.CONT.HOOK [ INDEX #index1,
                                        ICONS-KEY #icons1 ],
                           COMPS.FIRST.LOCAL.CONT.HOOK [ INDEX #index2,
                             ICONS-KEY #icons2 ] ],
    C-CONT.ICONS.LIST < info-str & #icons1 & [ IARG2 #index1 ],
            info-str & #icons2 & [ IARG2 #index2 ] > ].

add-icons-subj-comp-verb-rule := lex-rule &
  [ SYNSEM.LOCAL [ CONT.HOOK [ INDEX #index1,
                   ICONS-KEY #icons1 ],
           CAT.VAL [ SUBJ.FIRST.LOCAL.CONT.HOOK [ INDEX #index2,
                              ICONS-KEY #icons2 ],
                 COMPS.FIRST.LOCAL.CONT.HOOK [ INDEX #index3,
                               ICONS-KEY #icons3 ] ] ],
    C-CONT.ICONS.LIST < info-str & #icons1 & [ IARG2 #index1 ],
            info-str & #icons2 & [ IARG2 #index2 ],
            info-str & #icons3 & [ IARG2 #index3 ] > ].

delph-in / matrix

Information structure affixes work unfinished #494