IHTSDO / snowstorm

Scalable SNOMED CT Terminology Server using Elasticsearch
Other
208 stars 83 forks source link

Confusion regarding vaccine products #404

Closed bcorcoran closed 2 years ago

bcorcoran commented 2 years ago

Hi,

I'm using the latest version of Elasticsearch 7 and Snowstorm 7.6.0 on Ubuntu 20.04

In previous versions of Snowstorm (6.x) and the snomedct datafiles, I was able to search for "flu vaccine" and it would return results such as "Influenza Vaccine". Likewise, I could search "DPT vaccine" and receive all the concepts associated with DPT vaccines.

I understand the terms and concepts are always being updated, but is there a reason why the above no longer return any results at all?

Edit for clarification:

I used to search the "incorrect" way (appending "vaccine" to the search term and limiting by semanticTag on /browser/{branch}/descriptions), but now I am limiting the search area using ecl, for example:

endpoint: /MAIN/concepts (main = international edition. I have also tried US edition)

term = "flu"
ecl = <787859002|Vaccine product|
limit = 50

The above returns 0 results. If I change term to "influenza" it returns 31 results.

kaicode commented 2 years ago

Hi @bcorcoran, thanks for reaching out.

I think the different search results are related to content changes. We can test this theory by searching older releases of the International Edition using the Snomed International browser which allows selection of a release in the green dropdown.

Screenshot 2022-05-16 at 10 41 36

When searching using the term "flu vaccine" against the Jan 2020 International Edition we get a result with the "product" semantic tag.

Screenshot 2022-05-16 at 10 42 41

We also find that product when searching the 2020-01-31 release using the term "flu" and the ecl you suggested: https://snowstorm.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2020-01-31/concepts?term=flu&ecl=%3C787859002%20|Vaccine%20product%20(product)|

However in July 2020 in the International Edition the concept 46233009 |Influenza virus vaccine (product)| was made inactive with a historical association Possibly Equivalent to 836377006 |Vaccine product containing Influenza virus antigen (medicinal product)|:

Screenshot 2022-05-16 at 10 47 20

The concept inactivation indicator shows that the concept was made inactive because it was ambiguous. Probably because the term "FLU" is quite vague / non-specific.

It looks like the story is the same for the other example concept you gave, matching "DPT vaccine". If we enable matching active and inactive concepts in a browser search using the latest content we find that this concept has also been made inactive:

Screenshot 2022-05-16 at 10 51 19

Your searching technique looks good to me, it's just that the ambiguous content has been removed. Sorry to give you mixed news but I hope this answers your question.

Kind regards, Kai Kewley

bcorcoran commented 2 years ago

Unfortunate to hear that, but thank you for the clarification!

I will just have to document this for our users new to using SCT.