Proj: Design more contrastive editing toy problems

[g-simmons]

Possible Behavior Scores:

• Basically anything that can be measured from a piece of text. Candidates include:
  • Sentiment
  • Toxicity
  • Flesch-Kincaid
  • some quantiative measure based on POS
  • Train a model to look for a particular property (text is about some given topic, etc)
  • Class probabilities

Topological Expectations:

• concepts from network topology (bus, ring, mesh, etc.)

• intrinsic dimension (how many axes of variation) (maybe a dataset with two continuous axes of variation (e.g. sentiment and something else).

• Betti Numbers

• Noise? how much variation vs. axis of "real" variation

• [ ] Model-in-the-loop

• [ ] Topological expectations

[g-simmons]

@AbhiramBorra could you add a description of your suggestions from the last couple meetings here?

(or write a gdoc and add a link here?)