Improvement of lexical matches

[matentzn]

Issues with the current matching to deal with

• [ ] missed synonyms (bag of words matching could be better than relying on synonyms?): neuroleptic malignant syndrome -> Malignant neuroleptic syndrome not caught The semapv:UnspecifiedMatching part of your input does not fit into the table as it seems to be an additional piece of information that doesn't align with the columns provided. If you need this included in some way, please provide further instructions on how to represent it.
• [X] irrelevant categorical information or synonyms in brackets: Leprosy [Hansen’s disease], Pregnancy with abortive outcome (O00-O08)
• [ ] Broken encodings Waldenström macroglobulinemia - any smart idea how to handle with now access to the source? Here we need a smart way. What comes to mind is replacing broken chars with regex wildcards like Waldenstr.m macroglobulinemia.
• [ ] obsolete X -[skos:exactMatch]-> X we should match these despite their obsoletion
• [ ] disorder vs disorders (plural wordforms - do not manually implement, use some kind of NLP packages)
• [ ] other specified X, other unspecified X --[skos:broadMatch]->X

[nicolevasilevsky]

I have a list of mappings I didn't agree with here.

NV to do:

• [x] to do: I will add a summary of mappings I disagree with below.
• [x] decide if it is better to consider ICD10 synonyms as close match or exact match
• [x] review the 'yes' for exact synonyms

For ICD10 mappings

• before producing mapping suggestions, always run boomer (to ensure that a new ICD10 is not mapped to a Mondo class that already has an ICD10 mapping)
• synonyms in ICD10 are not exact synonyms, they are close matches
• if Mondo matches a synonym in ICD10, the Mondo term is more specific than the ICD10 term (ie MONDO:mondoIsNarrowerThanSource). ICD10 seems to be grouping terms that are more specific and includes them as 'approximate synonym's or 'inclusion terms'
• issue with lexical matching:
  • example: ICD10CM:A15.0 Tuberculosis of lung has 'inclusion term' tuberculous fibrosis of lung, which is an exact match to the label for MONDO:0021953. But we have MONDO:0006052 pulmonary tuberculosis, which should be an exact match to ICD10CM:A15.0.
• discussed on tech call, we do not want to map to 'other' terms in ICD10 or terms with 'not otherwise specified'.
  • @hrshdhgd preprocessing step in synonimizer: in ICD, if the label is other X, add broad synonym X

[nicolevasilevsky]

For orphanet mappings

• if there is a match on a more specific mapping predicate (broad match), prefer that to xref
• several mappings already exist in Mondo to Orphaent
  • in some cases, a new mapping is suggested but there is already an equiv mapping to another term in Mondo

[hrshdhgd]

• [ ] missed synonyms (bag of words matching could be better than relying on synonyms?): neuroleptic malignant syndrome -> Malignant neuroleptic syndrome not caught

---

This is untrue: doid.sssom.tsv line 24163

Here's a table representing the data you've provided:

subject_id	subject_label	predicate_id	object_id
DOID:14464	Neuroleptic Malignant Syndrome	oboInOwl:hasDbXref	ICD10CM:G21.0

Also icd10cm_mapping_status.tsv line 95175

subject_id	subject_label	is_mapped	is_excluded	is_deprecated
ICD10CM:G21.0	Malignant neuroleptic syndrome	True	False	False

---

The semapv:UnspecifiedMatching

That was the placeholder we decided to put there at the time. If there is some other class that makes sense , please suggest.

Broken encodings Waldenström macroglobulinemia - any smart idea how to handle with now access to the source? Here we need a smart way. What comes to mind is replacing broken chars with regex wildcards like Waldenstr.m macroglobulinemia.

This could be a solution. The question is where should this ending code lie?

>>> incorrect_string = "Waldenström macroglobulinemia"
>>> bytes_string = incorrect_string.encode('latin1')
>>> correct_string = bytes_string.decode('utf-8')
>>> print(correct_string)
Waldenström macroglobulinemia

obsolete X -[skos:exactMatch]-> X we should match these despite their obsoletion

I think it already does for a few. E.g.:

mondo_exactmatch_icd10cm.tsv	subject_id	subject_label	predicate_id	object_id	object_label	mapping_justification	mapping_tool	confidence	subject_match_field	object_match_field	match_string	comment
MONDO:0024297	obsolete nutritional or metabolic disease	skos:exactMatch	ICD10CM:E00-E90		semapv:UnspecifiedMatching						MONDO_MAPPINGS

disorder vs disorders (plural wordforms - do not manually implement, use some kind of NLP packages)

An example would help. I tried looking this up but didn't come across unmapped ones that had to be mapped.

other specified X, other unspecified X --[skos:broadMatch]->X

Again, an example would help.

@hrshdhgd preprocessing step in synonimizer: in ICD, if the label is other X, add broad synonym X

This has been done.

[matentzn]

@hrshdhgd can you move the entire content of this issue into a well-structured Google docs (headlines for each of the lexical-matching optimisations)? I think we should have all the different lexical optimisations discussed a bit and GitHub is terrible for this. Just post the link to the docs here, and I will answer to all your questions in there.