medizininformatik-initiative / Projectathon6-miracum1

5 stars 4 forks source link

Problems with the FHIR search queries when using the blaze FHIR server #5

Closed RaffaelBild closed 2 years ago

RaffaelBild commented 2 years ago

Hello,

unfortunately it seems that the Blaze FHIR server does not support FHIR search reference parameters. As a consequence, the first FHIR search query

Encounter?date=ge2015-01-01&diagnosis:Condition.code=I60.0,I60.1,I60.2,I60.3,I60.4,I60.5,I60.6,I60.7,I60.8,I60.9,I61.0,I61.1,I61.2,I61.3,I61.4,I61.5,I61.6,I61.8,I61.9,I63.0,I63.1,I63.2,I63.3,I63.4,I63.5,I63.6,I63.8,I63.9,I67.80!&_include=Encounter:patient&_include=Encounter:diagnosis

does not work as expected when using Blaze, because it cannot select Encounters based on ICD codes of Conditions they reference. More precisely, the query returns 0, even though the data definitely containes Encounters that meet the specified search criteria.

Is there any way to still do this analysis when using the Blaze server?

palmjulia commented 2 years ago

@alexanderkiel just to be sure: The above query doesn't seem work with the blaze. Is that because this kind of Search Parameter is not implemented or is something going wrong with the references in the server?

RaffaelBild commented 2 years ago

Exactly, it is because this kind of search parameter is not supported by Blaze. The references in the dataset used are correct. I've verified that the Query does work when using the IBM FHIR server using exactly the same dataset.

palmjulia commented 2 years ago

Okay, because I think we had a case somewhere where a reference query didn't work because there references weren't correctly resolved internally. Does the query work if you remove the Condition qualifier, i.e. Encounter?date=ge2015-01-01&diagnosis.code=I60.0,I60.1,I60.2,I60.3,I60.4,I60.5,I60.6,I60.7,I60.8,I60.9,I61.0,I61.1,I61.2,I61.3,I61.4,I61.5,I61.6,I61.8,I61.9,I63.0,I63.1,I63.2,I63.3,I63.4,I63.5,I63.6,I63.8,I63.9,I67.80!&_include=Encounter:patient&_include=Encounter:diagnosis ?

alexanderkiel commented 2 years ago

@RaffaelBild You are right. The reason is that Blaze doesn't support chained search parameters jet. So diagnosis:Condition.code or diagnosis:code won't work. However _include works.

palmjulia commented 2 years ago

Okay, that's good to know, thanks. @RaffaelBild is there any chance you could use another type of server? It seems like you have an IBM available? If not, the R-Code would have to be rewritten (yet again...) and that might take some time. As far as I can see at the moment one would have to download all Encounters and include their conditions and then filter the Encounters locally after the download. Depending on the number of Encounters on your server that might come at massive memory costs. Or I guess one could also download the Conditions and _revinclude the Encounters, but then the Encounters would still have to be filtered for the correct time frame after. Either way this would need additional programming by @NandhiniS08 and @mematt and I'm not sure if this is feasible right now because we are under a bit of time pressure.

RaffaelBild commented 2 years ago

Thank you all for the information. I do have an IBM available and will use this server for the analysis then.

palmjulia commented 2 years ago

Great, thank you both!

NandhiniS08 commented 2 years ago

Thank you @palmjulia for your quick response! @RaffaelBild Yes it would be great if you can use the similar type of server (IBM). Thank you!

FloSeidel commented 2 years ago

Hi, we are currently using Blaze as well. Unfortunately we are not having another server on hand :/

NandhiniS08 commented 2 years ago

@FloSeidel Is the FHIR search query also gives you 0 encounters?

mematt commented 2 years ago

Hi, thank you all @palmjulia , @RaffaelBild , @alexanderkiel, @FloSeidel and @NandhiniS08 for you inputs.

@FloSeidel did you have the opportunity to try out @palmjulia's suggestion regarding the query?

palmjulia commented 2 years ago

As @alexanderkiel wrote the Blaze currently doesn't support chaining at all, so neither Encounter?diagnosis.code nor Encounter?diagnosis:Condition.code would work. I am a bit at unsure what to do here now, it is quite unfortunate that none of the sites using blaze has tested the code until now. _include, _revinclude and _has work on the blaze as @alexanderkiel told me. So you could try to adapt the script yet again (maybe in a different branch) so that the request is based on the Condition (filtering for the correct codes with Condition?code=...) and then include the Encounters with _revinclude=Encounter:diagnosis. Then you'd only have to filter the Encounters for the correct date range in a second step. I don't know how many sites are exclusive Blaze users and whether you are willing to do another adaption like this. I would lock the master anyway so that Karo can send out the request to run the analysis to all other sites. Even if you decide to provide an alternative for the Blaze users, the results should be exactly the same in the end, just with a different downloading scheme.

FloSeidel commented 2 years ago

@FloSeidel Is the FHIR search query also gives you 0 encounters?

The initial query Encounter?date=ge2015-01-01&diagnosis:Condition.code=I60.0,I60.1,I60.2,I60.3,I60.4,I60.5,I60.6,I60.7,I60.8,I60.9,I61.0,I61.1,I61.2,I61.3,I61.4,I61.5,I61.6,I61.8,I61.9,I63.0,I63.1,I63.2,I63.3,I63.4,I63.5,I63.6,I63.8,I63.9,I67.80!&_include=Encounter:patient&_include=Encounter:diagnosis results in 0 (as expected).

Your adaption Encounter?date=ge2015-01-01&diagnosis.code=I60.0,I60.1,I60.2,I60.3,I60.4,I60.5,I60.6,I60.7,I60.8,I60.9,I61.0,I61.1,I61.2,I61.3,I61.4,I61.5,I61.6,I61.8,I61.9,I63.0,I63.1,I63.2,I63.3,I63.4,I63.5,I63.6,I63.8,I63.9,I67.80!&_include=Encounter:patient&_include=Encounter:diagnosis ends with a result != 0.

palmjulia commented 2 years ago

Careful, the second result is !=0 but still not correct, I think it just gives back the same number as Encounter?date=ge2015-01-01 would. As I said, neither of the two versions of chaining will work on the blaze.

FloSeidel commented 2 years ago

Yep. I can confirm, it's the same.

NandhiniS08 commented 2 years ago

I updated the code as per @palmjulia suggestion to download the resources without chaining and pushed them into a different branch blaze_update Branch link: https://github.com/medizininformatik-initiative/Projectathon6-miracum1/tree/blaze_update

@FloSeidel Is it possible for you to pull this branch and execute it? Thank you!

FloSeidel commented 2 years ago

Will do. I get back to you when it finishes.

alexanderkiel commented 2 years ago

@FloSeidel I've released the v0.17.0 of Blaze which supports chaining and is a drop-in replacement for the v0.16.x versions. So you can just try it on your existing data.