hapifhir / hapi-fhir

šŸ”„ HAPI FHIR - Java API for HL7 FHIR Clients and Servers
http://hapifhir.io
Apache License 2.0
1.94k stars 1.3k forks source link

Bulk-import Jpa-Server-Starter ID constraint error #5869

Open Loiotile opened 2 months ago

Loiotile commented 2 months ago

Most of the times we try to run a bulk-import on our unsorted data (resources kept inside .ndjson files, with multiple references to eachother ids that need to be maintained), we encounter the following error: "ERROR c.u.f.c.BulkImportCommand HTTP 500 : Job is in FAILED state with 22 error count. Last error: Too many errors: 4. Last error msg was HAPI-0550: HAPI-0825: The operation has failed with a client-assigned ID constraint failure. This typically means that multiple client threads are trying to create a new resource with the same client-assigned ID at the same time, and this thread was chosen to be rejected. It can also happen when a request disables the Upsert Existence Check". This error doesn't occur for smaller groups of data; also, in rare occasions the program run with same settings and data causing this error can end up successful. In our application.yaml have "autocreate_placeholder_reference_targets: true", in order to create placeholders for resources referenced before the resources themselves are actually populated, and we assumed that these placeholders could have been the reason behind the error.

Additional context: Use of hapi-fhir-CLI with localhost and PostgreSQL

jkiddo commented 2 months ago

I have the same observation when trying to do even small synthea-based imports running on H2.