IHTSDO / snowstorm

Scalable SNOMED CT Terminology Server using Elasticsearch
Other
182 stars 73 forks source link

Feature request; store expression used in EHR #87

Open waitingmyself opened 4 years ago

waitingmyself commented 4 years ago

Is this possible that store the expression associated with EHR in this index, and then query through ecl ? just like :{"patient":"XXXX","expressions":"119189000 |Ulna part| + 312845000 |Epiphysis of upper limb|: 272741003 |Laterality| = 7771000 |Left|"}

kaicode commented 4 years ago

Hi @waitingmyself, thank you for asking for something new and innovative.

What you are asking for sounds like a post coordinated expression repository. I think the scope of a terminology server like Snowstorm is to provide terminology services rather than storing any patient information, even just a patient identifier, but storing the expressions is in scope.

Snowstorm is not able to do this yet but let's talk about how it could work.

The way I could see this working would be for the EHR system to send an expression to Snowstorm and get back an SCTID identifier which would be stored in the patient record. Snowstorm would validate the expression and attempt to find an existing equivalent concept or expression with an existing identifier. If an existing concept or expression is found that identifier would be returned. If the expression is new a new identifier would be created and the expression persisted.

The ECL search could then match stored expressions in addition to concepts and return their identifiers. The EHR system could match the identifiers against the patient records.

Expression validation would be performed against the Machine Readable Concept Model reference sets to check that the attributes given are of an appropriate type and value to use with the stated parents.

I would also see value in Snowstorm classifying the expression to pick up additional parents or attributes before storing it in the semantic index. I used your expression to create a pre-coordinated concept in our test authoring platform and classification picked up an additional parent - 719462006 | Bone structure of left ulna (body structure) |. Inferred relationships should be captured in order for ECL to work at it's best.

How does this sound? Would this meet your requirements?

I wonder if other people would be interested in such a feature.

Kind regards, Kai

waitingmyself commented 4 years ago

Thank you for your reply @kaicode.

It is not a best practice to store case information in the terminology service system. Besides that, what you are talking about is exactly what I need.

Because creating and publishing a concept can be a cumbersome process, especially in EHR or knowledge bases, using post coordinated expression might be a good solution.

But for me, the difficulty is to retrieve these expressions, I noticed that your project has been able to parse the full ECL, so I want to try out the expression retrieval function with your project.

Anyway, thank you very much for your reply, I will continue to pay attention to your project.

Kind regards,

alexissmirnov commented 4 years ago

@kaicode:

I wonder if other people would be interested in such a feature.

Yes!

Storing post coordinated expressions in Snowstorm and querying it with ECL is exactly what I'm also describing in https://github.com/IHTSDO/snowstorm/issues/100

mertenssander commented 4 years ago

Hi @kaicode @waitingmyself , Great suggestion! I have been thinking a lot on how to achieve something similar to this, but for some reason never posted it here. I would be very interested in this!

We are working with several parties on optimizing patient data analysis using SNOMED, and one of the things I love about SNOMED is the ability to aggregate patient data using the descendants of a concept as a filter list. This already works great with ECL queries.

We ran into issues when we needed to determine the ancestors of an ECL query (of course this would actually be a postcoordinated concept), as you (preferably) need to know the ancestors of a concept when filtering a dataset in Power BI or the like - can go on and on on this, but I'll spare you. I did manage a workaround, but it is very time consuming, counter-intuitive and slow.

If I understand you correctly @kaicode , we would be able to create a postcoordinated concept or find the equivalent concept using this method?

kaicode commented 4 years ago

Thank you @alexissmirnov and @mertenssander for showing your interest in this use case.

Yes this is an appropriate use of Snowstorm. This year we will be pushing forward with better tooling support and guidance for post-coordination in SNOMED CT.

We have recently update the MRCM endpoints to support accurate authoring of post-coordinated expressions with the correct attribute types for the content domain and the ability to search within the correct range of an attribute to find a value. See GET /mrcm/{branch}/domain-attributes and GET /mrcm/{branch}/attribute-values/{attributeId}

At the moment there is no way to persist a post coordinated expression specifically. Having said that in the short term it seems acceptable to simply create a concept in the same way as you would if creating a pre-coordinated concept in an extension using POST /browser/{branch}/concepts.

The new concept will have to be classified in order to get inferred relationships which are used in ECL queries. Snowstorm currently uses an external Classification Service for this. Later in the year we will be adding incremental classification functionality, probably directly in Snowstorm. This will allow post-coordinated expressions to be classified within a few seconds which will speed up the authoring process.

I am also involved in a UK project which will be creating an open source user interface for creating post-coordinated expressions using the MRCM for the purpose of creating mappings from hospital code systems to SNOMED CT. Once that is available I will share that work here.

I am not yet certain what other features people want or will be required by the project I am involved in. For example authoring and lookup via SNOMED Compositional Grammar seems like a nice to have but most people seem to want descriptions with their post-coordinated content so more thought is needed there.

I'll be picking this all up in May.

Kind regards, Kai

kaicode commented 10 months ago

Back in April this year I released a new experimental Postcoordination feature in version 8.3.0 of the SnowstormX project. This is to accompany the SNOMED CT Practical Guide to Postcoordination that has been published for community feedback.