Wrong 'ValueString' for 'display' of CodeSystem concept

[vachonr]

Hello

I whish to retrieve the displayed term for the SNOMED concept 15122003. FSN : 'Algae (Organism').

When I do the following request https://snowstorm-fhir.snomedtools.org/fhir/CodeSystem/$lookup?system=http://snomed.info/sct&code=427623005 on the web. I obtain: {"resourceType":"Parameters","parameter":[{"name":"version","valueString":"http://snomed.info/sct"},{**"name":"display","valueString":"Algae"**},{"name":"active","valueBoolean":true},{"name":"property","part":[{"name":"code","valueString":"effectiveTime"},{"name":"valueString","valueString":"20020131"}]},{"name":"property","part":[{"name":"code","valueString":"moduleId"},{"name":"value","valueCode":"900000000000207008"}]},{"name":"designation","part":[{"name":"language","valueCode":"en"},{"name":"use","valueCoding":{"system":"http://snomed.info/sct","code":"900000000000003001","display":"Fully specified name"}},{"name":"value","valueString":"Algae (organism)"}]},{"name":"designation","part":[{"name":"language","valueCode":"en"},{"name":"use","valueCoding":{"system":"http://snomed.info/sct","code":"900000000000013009","display":"Synonym"}},{"name":"value","valueString":"Alga"}]},{"name":"designation","part":[{"name":"language","valueCode":"en"},{"name":"use","valueCoding":{"system":"http://snomed.info/sct","code":"900000000000013009","display":"Synonym"}},{"name":"value","valueString":"Algae"}]},{"name":"property","part":[{"name":"code","valueString":"parent"},{"name":"value","valueCode":"39685003"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"243564003"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"83279007"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"82114006"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"75805000"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"60986003"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"41745000"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"34503001"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"26913007"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"22884009"}]},{"name":"property","part":[{"name":"code","valueString":"child"},{"name":"value","valueCode":"3537008"}]}]}

{"name":"display","valueString":"Algae"} is correct. 'Algae' is the expected "valueString" for "display".

I have written a small python code which perform the following query http://localhost:8080/fhir/CodeSystem/$lookup?system=http://snomed.info/sct&code=15122003&_format=json. on my local SnowStorm (SnowStorm: version 7.3.0, elastisearch version: 7.7.0, Snomed release SnomedCT_InternationalRF2_PRODUCTION_20210731T120000Z).

The result of the query is : [{'name': 'version', 'valueString': 'http://snomed.info/sct'}, {'name': 'display', 'valueString': 'Algae, NOS'}, {'name': 'active', 'valueBoolean': True}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'effectiveTime'}, {'name': 'valueString', 'valueString': '20020131'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'moduleId'}, {'name': 'value', 'valueCode': '900000000000207008'}]}, {'name': 'designation', 'part': [{'name': 'language', 'valueCode': 'en'}, {'name': 'use', 'valueCoding': {'system': 'http://snomed.info/sct', 'code': '900000000000003001', 'display': 'Fully specified name'}}, {'name': 'value', 'valueString': 'Algae (organism)'}]}, {'name': 'designation', 'part': [{'name': 'language', 'valueCode': 'en'}, {'name': 'use', 'valueCoding': {'system': 'http://snomed.info/sct', 'code': '900000000000013009', 'display': 'Synonym'}}, {'name': 'value', 'valueString': 'Alga'}]}, {'name': 'designation', 'part': [{'name': 'language', 'valueCode': 'en'}, {'name': 'use', 'valueCoding': {'system': 'http://snomed.info/sct', 'code': '900000000000013009', 'display': 'Synonym'}}, {'name': 'value', 'valueString': 'Algae'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'parent'}, {'name': 'value', 'valueCode': '39685003'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '243564003'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '83279007'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '82114006'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '75805000'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '60986003'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '41745000'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '34503001'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '26913007'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '22884009'}]}, {'name': 'property', 'part': [{'name': 'code', 'valueString': 'child'}, {'name': 'value', 'valueCode': '3537008'}]}]

In this case I obtain 'Algae, NOS' in {'name': 'display', 'valueString': 'Algae, NOS'}. 'Algae, NOS' is not expected here but 'Algae'.

Something is wrong but I don't understand what.

Please could you help me ? Thanks.

René test.log : the log of my script with the launch of snowstorm. elasticsearch.log test.log elasticsearch.log

[kaicode]

Hi René,

Your local FHIR API query looks okay to me and I agree that the "Algae, NOS" term is not expected. This term is part of the concept but it is inactive. This suggests that the other terms for that concept were not imported correctly into your instance.

There are lots of errors in your elasticsearch.log like this:

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks

Apparently this suggests that there are multiple Elasticsearch instances running on that machine and they are conflicting with each other, see this Elasticsearch support discussion.

I would suggest shutting down Snowstorm and all Elasticsearch instances (restarting the machine might be the easiest way). Then try running the SnomedCT SNAPSHOT import again. Check the Snowstorm logs to be sure that the import completes successfully before trying your FHIR request again. Sorry I can't be more help from here!

Best of luck, Kai

[vachonr]

Thanks Kai

I'have shut down all elasticearch instances, and reinstalled elasticsearch 7.7.0 (x86). I have run SNOMED CT import without error. But I have still the issue.

Regards.

René test.log

[kaicode]

We could verify that all the content has imported correctly by checking the total numbers of concepts, descriptions and refset members.

In the July 2021 release there should be 485,977 concepts as shown in the total count: https://snowstorm.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2021-07-31/concepts?limit=1

1,529,295 descriptions: https://snowstorm.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2021-07-31/descriptions?limit=1

4,922,883 refset members: https://snowstorm.ihtsdotools.org/snowstorm/snomed-ct/MAIN/2021-07-31/members?limit=1

Do you have similar numbers? If you have much less than this then it's probably still an import problem. Make sure you delete the data directory while Elasticsearch is shut down to delete all the index data, reinstalling it may not have done that.

[vachonr]

Thank you

I'm sorry but I didn't manage to count the requested elements.

I haven't acces to the localhost via a web browser. I work with a remote server. A shell launchs elasticsearch and snowstorm. And then it launchs following python code:

" import requests

query = f"http://localhost:8080/fhir/CodeSystem/\ $lookup?system=http://snomed.info/sct&code=15122003&_format=json"

answer = requests.get(query).json()["parameter"]

print(f"\nResult of the query {query}:\n") print(answer) "

I didn't find the query which give the number of concepts, descriptions and refsets members.

Please could you help me ?

René

[kaicode]

@vachonr Thank you for the snippet of Python code. That has allowed me to reproduce the problem you are facing.

It appears that there is a bug in the Snowstorm FHIR API; when selecting the display term there is no default dialect. If no dialect is specified in the request the API just returns the first term it finds rather than a preferred term.

We will get this fixed. In the meantime please workaround this issue by including the displayLanguage parameter in the request like this:

requests.get("http://localhost:8080/snowstorm/snomed-ct/fhir/CodeSystem/$lookup?system=http://snomed.info/sct&code=15122003&_format=json&displayLanguage=en-us").json()["parameter"][1]

That correctly returns:

{u'name': u'display', u'valueString': u'Algae'}

I think the reason it works when making the request from a web browser is that additional Accept-Language headers are included in the request and a display language is determined from these.

[vachonr]

Thanks @kaicode

I will use the request including the displayLanguage parameter until the issue is fixed. Regards.

René.

[kaicode]

If it's okay with you, I would like to leave this issue open until the fix is released. That helps us and others be aware of the issue and the progress of the fix.