search

AndreasSko / go-jwlm

A command line tool to easily merge JW Library backups, written in Go.
MIT License
71 stars 4 forks source link

Optimize detection for migration to nwtsty #108

Closed AndreasSko closed 2 years ago

AndreasSko commented 2 years ago

The current approach of determining the need for migrating from nwt to nwtsty (introduced with https://github.com/AndreasSko/go-jwlm/pull/46) is using a heuristic, which basically checks if one side has significantly more nwtsty entries for a specific language than the other side. This heuristic worked quite well, but can fail if the difference between both sides is not high enough. This can lead to collisions of UserMarkGUIDs:

Could not create SQLite database for exporting: [...] UNIQUE constraint failed: UserMark.UserMarkGuid

This introduces a new approach, which will check both sides for the same UserMarkGUID. If it finds a common GUID, it will check if the location of one UserMark is in nwt, while the other one is in nwstsy. If this is the case, it indicates the need for migrating to nwtsty, as obviously one side has already been migrated.

The logic for the migration is now bundled in kind of a "PreMergeHook", which bundles necessary functionality for preparing the databases for merging.

Cleanup userMark duplicates after nwtsty migration

The migration from nwt to nwtsty is working properly for "simple" locations, e.g. Bible chapters. However, the locations for appendices have DocumentIDs, which are not easily migratable (we would need to find the documentID corresponding to the nwstsy). Therefore, we skip those locations. Skipping the locations, however, introduces the risk of UserMark collisions again, as the merger is using the locationIDs for finding those. As this is a very limited edge-case, I decided to simply remove the duplicate from the old nwt side completely. This shouldn't be a problem, as markings on the nwstsy can be considered newer.

closes https://github.com/AndreasSko/go-jwlm/issues/105 closes https://github.com/AndreasSko/go-jwlm/issues/97 closes https://github.com/AndreasSko/ios-jwlm/issues/36

coveralls commented 2 years ago

Coverage Status

Coverage increased (+0.2%) to 86.195% when pulling 10ec326f854858154a1e2fd9d5b6368b82efc7e4 on migrateBible into c192d0b315b9fee77974ac38c13697fdb84f1a86 on master.

AndreasSko commented 2 years ago

ToDo: Gomobile

AndreasSko commented 2 years ago

ToDo: