concept_ancestor has non-standard concepts

[dimshitc]

this query returns 243 rows

select * from v20240229_omop.concept_ancestor a where exists (select from v20240229_omop.concept c where c.standard_concept is null and (c.concept_id = a.ancestor_concept_id or c.concept_id = a.descendant_concept_id)) limit 1000

[Alexdavv]

Hi @dimshitc,

Great catch! We didn't see the elephant in the room at least from 2021. More likely from the 2020 ATC release, or from the first RxNorm release after that, to be precise. The problem is RxNorm/RxNorm Extension are living their own lives with a great list of changes, while we've been trying to make ATC more stable to reduce its maintenance burden. Here is the result. But we already know that ATC and the entire drug Domain needs to be overhauled. We're actively seeking the resources for that.

In the meantime:

• The problem is 18 non_Standard RxNorm/RxE concepts that appear in the concept_ancestor table. In total, they form 243 hierarchical links, exclusively from ATC way down to RxNorm/RxE.
• With a bit different composition and numbers (47 concepts/223 links in v20210902, 16 concepts/223 links in previous v20230831) they're there for a while.
• Ideologically, it doesn't look lovely and contradict to vocabulary principles. Practically, I don't think it could affect any ETLs, studies or performance of the systems. We have other problems in the Drug domain that could practically affect the studies, e.g. missing ancestral links (github/forum) or potential others.
• we'll fix them. Wel'll definitely be able to apply a small shortcut fix on the next release in August, and will plan for a proper fix with the ATC overhaul.

[dimshitc]

yeah, it's a very minor problem from the user perspective. I believe it originates from ATC postprocessing.