A `synonym` reference type

[jkomoros]

That points from a concept to another, and denotes that the entitled concepts are synonyms for one another.

The from and the to must both be concept types.

The relation should be symmetrical. How to enforce that?

Blocks on #399

When such a relation is set, the index terms for cards that point to a concept should "smear" to include the other concepts in the synonym group, once per time the primary concept shows up. (The smeared concepts should be ranked a bit lower)

[jkomoros]

• [x] A fromCardAllowList for reference types. Checked in data layer and also filters what types of references can be added in the UI
• [x] A 'synonym' reference type
• [x] Suggested concepts should not show items with synonym either
• [x] An 'Example-of' reference type
• [x] Syonym and reference-type get their own sections in info panel. (Or should they be small style reference blocks on the card itself?)
• [x] Create snonymMap
• [x] Wire through synonymMap into nlp pipeline same way
• [x] wordCountsForSemantics pretends it saw any synonym expansions
• [x] Should backported text from concept cards to referencing cards directly bring in the AKA? ... No, the synonym map handles that.
• [x] suggestedConcepts only needs one of a synonym class. In particular, it should skip fingerprint items that aren't actually on the card. Fingerprints grow a itemsFromCard -> map[word]true method, which is a factored out version of the prettyItems, and skips over derived fields, and also used as the set to iterate over in suggestedConceptReferencesForCard.
• [x] Memoize itemsNotFromCard if necessary
• [x] Synonym matches should count for, say, 0.75 of a full match
• [ ] Make sure all of the synonym machinery handles multi-word concepts correctly
• [x] (Nearly) everywhere that uses REFERENCE_TYPE_CONCEPT should instead use the config.conceptReference
• [x] should fingerprint.itemsFromConceptReferences also count items via an example-of or synonym reference?
• [ ] synonymMap does transitive expansion of synonyms (see TODO in synonymMap)
• [x] title_alternates gets a delimiter character in config (make sure that it renders ok)
• [x] title_alternates gets a help description about delimiters
• [x] AKA section rendered on cards looks better
• [x] Cards get a title_alternates that renders as a textarea in editor for concept cards
• [x] title_alternates are included in synonymMap
• [x] Concept cards render out AKA list on card
• [x] Does backporting of concepts work for example-of and synonym?
• [x] suggestedConceptReferencesForCard should skip references that are already fully contained by an accepted reference (this is actually party of #399)
• [ ] Make sure that AKAs that are strict supersets or subsets of a given term are handled correctly. E.g. Cynefin is an AKA for Cynefin framework
• [x] Fix vertical alignment issue for condensed reference blocks on cards, especially important now that they're in the sub-title area
• [ ] A lot of what were tags are now concepts. What's the difference? Should all concepts automatically be considered tags, too?
• [x] Make sure the synonym map includes title alternates (it should)
• [x] Make sure fingerprints.itemsFromConcepts should check the synonym map when deciding if it is from a concept
• [x] Concepts map should include title alternates
• [x] Example-of should also show up directly on a card
• [x] Some way to cache filters that use self ID, since we use so many of them. Maybe have active card be passed in? And make it so than when you navigate to a collection, that version includes self automatically. Hopefully that would allow more caching and faster navigation between cards. Now tracked in #431.
• [ ] Title alternates should show up in card thumbnail?
• [ ] If a card references a card as a concept and it references it, it will show up twice in the info panel "Cards that link here"
• [x] Syonyms, examples of, and aKA should all render on one line underneath title
• [x] Have opposite-of? Would count as a concept reference. There are a number in production that use see-also to a concept card now for this
• [x] In production, for levers card, super-linear (synonym of compounding) should be suggested, but instead linear is. Actually this appears to be a more general problem where suggested concepts don't appear to look for synonyms of concepts.
• [x] suggestedConceptREferencesForCard needs more work to deal with title alternates. normalizdConcepts should probably also return title alternates for example. and fingerprint.itemsFromConceptReferences likely needs to be expanded to also work for title alternates
• [x] Accepting 'compounding' (alt: super-linear') should shadow a recommendation for 'linear' on a card
• [x] When saving a concept card, title alternates also needs to be checked for non overlap
• [ ] Opposite-of should be changed to be 'in-contrast-to'. Maybe it's OK for it to just be changed in the way it's displayed to users.
• [ ] Actually change the 'In contrast to' relation (currently called 'opposite-of') to 'in-contrast-to' (requires maintenance task)
• [ ] Should 'example-of' relations have a synonym style relation, so any occurance of the example also pretends like it references the main one, too?
• [ ] Some way to have a disambiguation (e.g. open -> open-minded vs open-> open ecosystem. Or environment as in context vs environment as in nature)
• [ ] If you have one concept reference to another card, you shouldn't be able to have a sub-type (e.g. example-of).
• [ ] Make the notion of sub-types of references more general machinery (e.g. a custom filter e.g. concept-references, making sure that multiple of the same fundamental type aren't added)
• [x] Cards that include both a concept name and its synonym lead to double suggestions (which breaks tag-list in non-obvious ways). Introduced in 0e27d0466afc582f3d84d96becaf5888e544d4a6
• [ ] If a card links to a concept, but calls it one of its synonyms and never its 'proper' name, then the card link in the info panel would ideally show the synonym text as well (perhaps in parentheses)
• [ ] wordCountsForSemantics undercounts direct ocncept matches. It should do the same sum of organic + boost that synonyms does now
• [x] Working notes titles will now be dominated by concepts. It should filter out items that aren't literally from card. Maybe word clouds should do that too, or hae the not from card items be shaded subtle?
• [x] A card's reference blocks should update when the editing card's references change
• [x] Definitely need SOME way to figure out that e.g. 'complete stance' means meta-rationality. The concept-references are hidden-card-links would work, as would info in the word cloud for that word, as would the concept reference listing all of the ways the card referenced it. See #434 for adding that
• [x] Ideally the AKA list and synonym list would be rendered in the same way on a card as a combined list
• [ ] A derived indexing field that includes synonyms for every word on the card (ideally skipping ones that are on the card elsewhere). the fingerprint.itemsFromCard learns to skip that indexing field (that field can't just be a derived field because we do really want to index it in other parts of the pipeline). That will allow cards to show up for searches for synonyms
• [ ] Consider a mechanism to normalize every synonym group back to its canonical representation for the group, and create an nlp field that is deSynonymed that queries can be done and compared to, so everything can seamlessly pretend that the synonym is actually there in the card (kind of)
• [ ] system and agent shadow one another ins uggested concepts because they are synonyms, but not once you select the other. Ideally they wouldn't shadow each other during concurrent suggestions, or in sequential suggestions.
• [ ] A 'metaphor-for' concept type?
• [ ] Some way for a concept to suppress connections for words that stem to the same thing. Like 'useful' is wreaking havoc on suggestions because it reduces down to 'us'. The current way of handling that is just a manual stemming override.
• [x] Subtitle line should be able to wrap, for cards like 'force of gravity'
• [ ] Cards with alternate titles that include common words like 'system' (like 'clock speed') eat suggested concepts too aggressively, like 'system'
• [x] Some way to handle polarities directly on one card, where you want to represent both sides at once without creating both cards. Maybe some way to express a negative title alternate? Most of the pipeline treats it just as a synonym, but the UI presents it as a 'In contrast to' without another card to reference (yet). Maybe have items that are prefixed with a '-' mean 'negative'. Ideally when you do create that opposite card, you could identify cards that were pointing to it, but now are pointing to the wrong card. A mechanism to find cards that HAVE a concept but don't seem to actually contain the text? Ideally that would only capture things that USED to have the text but don't now.
• [ ] Tighter line height when the subtitle line wraps, like force of gravity concept card
• [x] Should 'Synonym-of' be 'interchangeable with' in the UI? since like system and agent aren't actually synonyms.
• [ ] Accepting a 'incentive' for card c-637-cbf943 shadowed 'intrinsic motivation' as a suggested concept despite the card literally using that terminology. Another example of effect noted in the next todo item? (no, this one is an oddity: fullyNormalizedString on an already fully normalized string makes intrins go to intrin and not match)
• [x] Now that agent and systems are synonyms, cards that explicitly use both terminology won't have the synonym suggested... but they should.
• [ ] Make synonyms be reciprocal somehow (maintenanc task on production)
• [x] Cards that point to another card as synonyms should also get the expanded synonym terms in their indexing (but not directly in any included runs)
• [x] make sure that cards that point to a synonym of a given card don't get that card suggested as a suggested concept (unless the card ALSO directly includes the concept text)
• [x] Cards should have some way of listing synonyms directly on themselves. E.g. Surfing a gradient might include other alternates directly configured/edited on self. And then the synonym reference type basically just adds to those automatically

[jkomoros]

A few things to think about with synonyms.

In some cases, it's a bidirectional synonym. In some cases it's a one directional (e.g. an example-of) relation. Some relations are weaker than others.

Should the ngrams be copy/pasted everywhere to transitive relation cards, or should each ngram in the synonym class be treated as expanding to the same ngram that represents the class?

There are some cases where you want to mark a synonym of a card but don't want to have they synonym be its own card. Maybe have a card.title_alternates that are back ported?

[jkomoros]

Sometimes you want there to be a key card for the synonym group that they all are based off of.

[jkomoros]

We could make it so every snonym group reduces down to a given particular word, and any time any of the alts are encountered they all reduce down to that word. But sometimes you do want the original wording. Maybe an extra normalized value in card.nlp that's synonyms removed?

[jkomoros]

The synonyms should start out just affecting wordCountsForCard, pretending that the synonym expansion words are there.

Each card can get a title_alternates strings. And for now just have it be represented in UI as a textarea where each line is an alternate title. (Joined and then split to consider if they're changed)

Then we calculate a synonymMap from concept cards that is a map of word => array of synonym expansions. By default that map would be just a join of title_alternates and any backported card titles it pointed to via synonym references. Then, it does that expansion a few times until it settles, so you get synonym expansions from tarnsitive cards. The synonym map would be passed around throughout the pipeline in the same way that importantNgrams is.

Later, there's a text field for indexing that takes any words in other fields and expands their synonyms using the synonym map, so at least they'll match a little bit if you search for that word, although they won't match the

And then maybe later we figure how to reverse a given word to its normalized version, and then have an extra nlp property on card of deSynonmed . That map would have to make sure that each one reduced to precisely one. And then queries would be normalized that way and queries would look over that text property for matches. (Although ideally with a boost for cards that actually match)