medic / cht-sync

Data synchronization between CouchDB and PostgreSQL for the purpose of analytics.
GNU General Public License v3.0
3 stars 5 forks source link

Enhance integration tests to ensure accurate data transfer #81

Closed lorerod closed 1 month ago

lorerod commented 5 months ago

During integration testing, verify data accuracy from CouchDB to Postgres to ensure the model reflects the source data. Additionally, rename existing End-to-End tests to Integration tests to reflect their purpose.

What else to test?

Initial Data Sync: Verify that data from CouchDB is correctly transferred to PostgreSQL during initial sync (Data in PostgreSQL should exactly match the data in CouchDB). Continuous Data Sync: Ensure continuous syncing works correctly when data is updated or added in CouchDB. Incremental changes are synced properly without reprocessing the entire dataset. Conflict Resolution: Ensure that conflicts (e.g., same document updated differently in multiple places) are handled properly during the sync process. (Not sure yet how this could happen) Large Data Volume: Test the system’s ability to handle large volumes of data. Error Handling: Validate the application's ability to handle and recover from errors during the sync process.

Pending tests: Deletion Sync: Confirm that document deletions in CouchDB are correctly propagated to PostgreSQL.

lorerod commented 2 months ago

Okay, this issue was partially addressed in feat(#112): drop support for multiple copies of every document and 112 handle cascade deletes:

lorerod commented 1 month ago

Added the remaining scenarios in chore(#81): enhance e2e tests