isi-vista / adam

Abduction to Demonstrate an Articulate Machine
MIT License
10 stars 4 forks source link

Output and display for affordances #1145

Closed spigo900 closed 2 years ago

spigo900 commented 2 years ago

We need to be able to display the affordances produced by #1111 and #1129 in the UI. Specifically, I think we want a list of text descriptions like 'can be SLOT2 in "SLOT1 throws SLOT2"', 'can be SLOT2 in "SLOT1 eats SLOT2"', et cetera. This requires some changes:

  1. On the ADAM system side:
    1. To IntegratedTemplateLearner, generally in the area of describe_common()
    2. To TopLevelLanguageLearnerDescribeReturn to include the new features.
      1. Add a semantic node -> feature mapping giving the text descriptions, similar to what we have for features. The keys are objects and the values are sequences (lists/tuples) of descriptions for affordances that the object has.
      2. Populate this mapping in one of the helper methods for describe_common().
      3. Generating the description output (as described at the top of this issue) requires knowing something about the template that the affordance comes from and the slot the affordance is focused on (SLOT2 in this case).
    3. To YAMLLogger, so that it outputs the affordances for an object. These should be minor, and similar to what's already there for (extracted visual) features.
  2. On the UI side:
    1. (_No change: The Flask backend can stay the same, since it simply passes along the output_language key._)
    2. To the UI, so it displays the affordances. This is a few small changes -- the most annoying part is how to display things. See the section below.

See #1141 for the "mapping affordances" API used to get the affordances.

Angular UI changes

Four small changes needed:

  1. Purely for typing reasons: The interface/s in selector-parent.component.ts, specifically LanguageResponse needs a new attribute, say derived_features, which should be a new interface DerivedFeatureResponse that is basically the same as FeatureResponse.
    1. The actual code doesn't need to change at all since (like the Flask backend) we're just passing along the output lanugage.
  2. object-results.component.ts needs to change to accommodate the new type of features, specifically makeLinguisticObject. This should be straightforward.
  3. object-results.component.html needs to change to implement the display. This should be easy -- almost a copy of the "features" section.
    1. We do want slightly different formatting to distinguish the extracted features from the things (affordances) we inferred based on those extracted features. The details of the change should go in the CSS, though.
  4. object-results.component.css probably should change to implement the formatting changes. (Italics? underline?).

The visual changes the least certain part. I think we will want to produce say four different demos of what the display could look like, review/discuss those, and go from there.