Open witash opened 2 weeks ago
The 2 * SYNC_INTERVAL
from the SYNC_PERIOD
is because there is a gap between when a resource is created in one system and when it is created in another system. So resources created near the boundary of SYNC_PERIOD
may appear in the query for one system, but be absent from the other, causing the sync to try and create them again.
So the idea was to add a window where a resource in one system that is not in the other is not created. It's not a good solution because creation times for corresponding resources in both systems can be different by much more than 2 * SYNC_INTERVAL
if the sync wasn't running or there was an error during the initial sync and it was resolved later.
One fix for this could be to make sure that the FHIR server doesn't allow duplicate resources, based on id or any of the other identifiers
: OpenMRS ID
, CHT Document ID
So if a resource appeared in the comparison list for OpenMRS but not FHIR, but was actually in the FHIR server, the sync would still send a creation request. But it wouldn't matter, it would get a 40X error and not do anything.
@witash While working on the openmrs-mediator
branch the openmrs
channel is not created. When checking through the logs I see this logged twice for the two mediators but the OpenMRS mediator does not have a default channel like the default mediator (see screenshots below)
I manually created the openmrs
channel to move forward but then I got a 400 error when sending a patient to OpenMRS (see logs below). I have pushed some of my work to the 141-openmrs-duplicate-resources
branch but I have left out a bunch of debugging logs I had added. The branch adds a defaultChannelConfig
to the OpenMRS mediator, although it didn't work as mentioned above, and fixes some scripts in the startup shell script.
2024-10-21 07:40:59 2024-10-21 04:40:59 INFO: Comparing Patient
2024-10-21 07:40:59 2024-10-21 04:40:59 INFO: Incoming:
2024-10-21 07:40:59 2024-10-21 04:40:59 INFO: Outgoing: d1fdf74a-0349-4133-bdee-866b14ad7b79
2024-10-21 07:40:59 2024-10-21 04:40:59 INFO: Sending Patient d1fdf74a-0349-4133-bdee-866b14ad7b79 to OpenMRS
2024-10-21 07:40:59 2024-10-21 04:40:59 ERROR: Request failed with status code 400 AxiosError: Request failed with status code 400
2024-10-21 07:40:59 at settle (/app/node_modules/axios/dist/node/axios.cjs:2019:12)
2024-10-21 07:40:59 at BrotliDecompress.handleStreamEnd (/app/node_modules/axios/dist/node/axios.cjs:3135:11)
2024-10-21 07:40:59 at BrotliDecompress.emit (node:events:529:35)
2024-10-21 07:40:59 at endReadableNT (node:internal/streams/readable:1400:12)
2024-10-21 07:40:59 at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
2024-10-21 07:40:59 at Axios.request (/app/node_modules/axios/dist/node/axios.cjs:4287:41)
2024-10-21 07:40:59 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
I will pick this up from the bug and fix the actual issue with the duplicates once I am back on Monday (28th of this month) and have the defaultChannelConfig automatic setup be a separate issue.
setting up the defaultChannelConfig is in this branch https://github.com/medic/cht-interoperability/pull/139 the 400 is probably because of #140 . for e2e-tests the workflow adds identifier types automatically now (in https://github.com/medic/cht-interoperability/pull/139) but for manual testing, they have to be added manually (or start everything with the e2e pipeline and leave it running)
Hopefully by the time you're back we'll have these merged to openmrs_mediator
OpenMRS sync currently subtracts the
2 * SYNC_INTERVAL
from theSYNC_PERIOD
to decide how long ago to sync. So if it's syncing 1 hour ago every minute, it only syncs resources which were updated in the last 58 minutes.This works around an issue where it was creating duplicate resources based on resources it had already created, but it does not really fix the core issues and is very confusing.
This workaround should be removed, and duplicate resources should be prevented by fixing the bug in the code that is supposed to be checking unique keys.