search

fastenhealth / fasten-onprem

Fasten is an open-source, self-hosted, personal/family electronic medical record aggregator, designed to integrate with 100,000's of insurances/hospitals/clinics
GNU General Public License v3.0
1.53k stars 83 forks source link

[Bug]: 403 errors in source sync #510

Open dwradcliffe opened 2 weeks ago

dwradcliffe commented 2 weeks ago

Contact Details

No response

What happened?

In my logs I'm seeing some failures while syncing with one of my (real) sources. I'm definitely able to get some data but it seems like a bunch is missing.

Flavor

Docker

Version

main#c5d7717

Relevant log output

skipping resource (ServiceRequest/xxxxxxxx), request failed: An error occurred during request https://sehproxy.stelizabeth.com/arr-fhir/HRH/api/FHIR/R4/ServiceRequest/xxxxxxxx - 403 - 403 Forbidden []
AnalogJ commented 2 weeks ago

This is actually expected. Fasten will attempt to extract and request all FHIR Resources that are referenced in your record. However most EHRs will only provide the medical record data that is legally mandated - USCDI Core v3

So any FHIR resource types that are not mandated by USCDI Core v3 will result in 403s (or other error messages). This isn't usually a problem, because USCDI Core includes all the "clinically" relevant medical records.

Having said that, because we're so permissive of errors (since we expect them), we may be inadvertently ignoring errors from FHIR record requests that should be retried -- I don't have a good solution here, open to ideas

dwradcliffe commented 2 weeks ago

Ah interesting (and frustrating). Is there a full list of types that are required? We could check against the list when ignoring the errors.

AnalogJ commented 2 weeks ago

Yes and no. The USCDI standard has a specified list of FHIR resources that we can check against, but EHRs & health systems can voluntarily support additional FHIR types -- they can choose to send back medical records data that they aren't mandated to share

dwradcliffe commented 2 weeks ago

I guess since it will retry next sync this isn't a huge problem, but we might just never know something is missing.

I think I would still hardcode a list of the USCDI standard resources so that we at least report errors if any of them fail. Can you point me to that list?

AnalogJ commented 2 weeks ago

Here's the list of FHIR resource types in USCDI core - http://hl7.org/fhir/us/core/2021Jan/profiles-and-extensions.html

Fasten-sources also has a list of Resource Types that we use for searching - https://github.com/fastenhealth/fasten-sources/blob/main/clients/internal/base/base_client.go#L71-L97

However, you'll notice that some are commented out, which is because this list is used for searching against the api, and some resource types like Location cannot be searched by Patient ID

There's alot of layers to this onion.