search

onc-healthit / inferno-program

Archived source code for the Inferno Testing Tool and the Program Edition set of tests. No longer maintained.
https://inferno.healthit.gov/
Apache License 2.0
38 stars 12 forks source link

DiagnosticReport search validations only check the first entry in CodeableConcept on responses (FI-1406) #412

Closed cooperthompson closed 2 years ago

cooperthompson commented 2 years ago

Subject of the issue When Inferno does code and category searches on DiagnosticReport (labs), it looks like you are validating that the code/category in the response matches what you searched on. However, it looks like you are only checking the first entry in the CodeableConcept, rather than any entry.

Your environment

Steps to reproduce Run USCDRLRR-03 and USCDRLRR-04 on a server returning DiagnosticReport resources that have multiple values in the code and category properties, where the searched value is not the first value in the list.

Include:

Expected behavior Tell us what should happen. When Inferno does a code or category search, it should check if the searched code is in any of the entries in the returned response, not just the first one.

Actual behavior Tell us what happens instead. Include screenshots if possible. Inferno looks like it is checking only the first entry in the CodeableConcept when validating that a returned resource is correctly included based on code/category search criteria.

Here is the DiagnosticReport search response bundle that is causing the errors: dxrpt.json.txt

image

cooperthompson commented 2 years ago

Also, should the comparison for search value to response content be case insensitive for tokens? Case sensitivity for tokens is talked about in the token definition, and it isn't really absolute. It does refer to CodeSystem.caseSensitive which references Postel's Law, which implies that a client should accept codes in any case.

arscan commented 2 years ago

Our logic should already check across all CodeableConcepts, and Codings within those CodeableConcepts. It seems like this is a case sensitivity issue as @cooperthompson brought up.... What are you returning in the first search (patient+category) for categories? We are using that to create searches for later tests. Are you returning a category of Lab (maybe in addition to LAB) in the first search? If that is the case, we then use that category to search in a later test.

Regarding whether or not we should be checking case sensitivity... it seems like by default we should not be failing due to incorrect case, unless we are very sophisticated and look in the CodeSystem.caseSensitive (which we aren't). Though, in the case of CodeSystem/v2-0074 it appears that it technically is case sensitive on equality checks, so http://terminology.hl7.org/CodeSystem/v2-0074|Lab is not technically correct? But maybe that code, in this case, is part of a different CodeSystem? We'd have to see the response to the first test to know for sure.

Thanks for helping us sort this out!

arscan commented 2 years ago

As a follow-up to the case insensitivity issue -- on searches it seems like we should allow servers to search case-insensitively on codes, and potentially return more resources than we'd otherwise expect. So we'd want to loosen our tests up to allow that. But I'm not sure that's the only thing going on here, because that doesn't explain 04. Part of the issue is that it looks like the output is trying to render as markdown in the interface in here, so we may not have all the information we need in the screenshot. It would be helpful if you could provide the first resource returned in the first search (assuming it has a code too), and/or the search parameters used in 04.

cooperthompson commented 2 years ago

I'd attached the dxrpt.json.text in the initial post, which has the full Bundle we return. Did you need more than that?

yunwwang commented 2 years ago

@cooperthompson The attached Bundle has one resource DiagnosticReport/ejDJxxxxx which does not match the resource id DiagnosticReport/ehgTxxxx in the screen shot.

Can you provide us the first several resources in the USCDRLRR-01 search results?

cooperthompson commented 2 years ago

Ah - sorry about there. Here is the Bundle with the both resources that errored in USCDRLRR-03 and USCDRLRR-04: dxrpt-01.json.txt

Also, since some of our IDs look similar (have the same starting characters), here is the text content from the screenshot for easy ID copy/pasting:

USCDRLRR-03: Server returns valid results for DiagnosticReport search by patient+category+date. results... category in DiagnosticReport/ehgTlC34iP1zQz5t8vX.iWi30fLkNXbeDvQD8zhG2oH43 ([#<FHIR::CodeableConcept:0x000055a38b7f9200 @id=nil, @extension=[], @coding=[#<FHIR::Coding:0x000055a38b7f80a8 @id=nil, @extension=[], @system="urn:oid:1.2.840.114350.1.13.861.1.7.10.798268.30", @version=nil, @code="DKF Lab", @display=nil, @userSelected=nil, @source_hash={"system"=>"urn:oid:1.2.840.114350.1.13.861.1.7.10.798268.30", "code"=>"DKF Lab"}>], @text="DKF Lab", @source_hash={"coding"=>[{"system"=>"urn:oid:1.2.840.114350.1.13.861.1.7.10.798268.30", "code"=>"DKF Lab"}], "text"=>"DKF Lab"}>, #<FHIR::CodeableConcept:0x000055a38b7f2900 @id=nil, @extension=[], @coding=[#<FHIR::Coding:0x000055a38b7f1ff0 @id=nil, @extension=[], @system="http://terminology.hl7.org/CodeSystem/v2-0074", @version=nil, @code="LAB", @display="Laboratory", @userSelected=nil, @source_hash={"system"=>"http://terminology.hl7.org/CodeSystem/v2-0074", "code"=>"LAB", "display"=>"Laboratory"}>], @text="Laboratory", @source_hash={"coding"=>[{"system"=>"http://terminology.hl7.org/CodeSystem/v2-0074", "code"=>"LAB", "display"=>"Laboratory"}], "text"=>"Laboratory"}>]) does not match category requested (Lab)

USCDRLRR-04: Server returns valid results for DiagnosticReport search by patient+code. results... code in DiagnosticReport/ehgTlC34iP1zQz5t8vX.iWjzhzeqZV96Rs2TzU7imZNw3 ([#<FHIR::CodeableConcept:0x000055a38ab5d488 @id=nil, @extension=[], @coding=[#<FHIR::Coding:0x000055a38ab5c498 @id=nil, @extension=[], @system="urn:oid:2.16.840.1.113883.6.12", @version=nil, @code="92313025", @display="CULTURE, GC", @userSelected=nil, @source_hash={"system"=>"urn:oid:2.16.840.1.113883.6.12", "code"=>"92313025", "display"=>"CULTURE, GC"}>], @text="CULTURE, GC", @source_hash={"coding"=>[{"system"=>"urn:oid:2.16.840.1.113883.6.12", "code"=>"92313025", "display"=>"CULTURE, GC"}], "text"=>"CULTURE, GC"}>]) does not match code requested (urn:oid:1.2.840.114350.1.13.861.1.7.2.696580

arscan commented 2 years ago

Yes, it looks like USCDRLRR-03 failed because of how we are treating case insensitive search matches on codes. I think its reasonable for our tests to allow that if it happens to come up.

USCDRLRR-04 appears to be a separate issue. I think we are searching by patient=exkr5BRhovpIY4hqjnmTPNQ3&code=urn:oid:1.2.840.114350.1.13.861.1.7.2.696580|52126 (you could confirm by looking at the requests performed in the "results..." popup) and yet you are returning a DiagnosticReport that only contains the following in DiagnosticReport.code: urn:oid:2.16.840.1.113883.6.12|92313025. Are those codes considered equivalent in your system?

cooperthompson commented 2 years ago

Ah - sorry. I incorrectly grouped USCDRLRR-03 and USCDRLRR-04. The code searching is something we are working on fixing on our end. There is a philosophical question there (is a server required to support searching for all code/code systems it returns in the resource?). But I don't mean to chase down that question here - that'd probably need some work in the US Core definition of mandatory search params.

So for the purpose of this ticket, you can ignore the USCDRLRR-04 part. Sorry about the mix-up - hopefully you didn't spend too much time chasing that red herring.