norrisng-bc commented 3 days ago

Description

When syncing, if a object not currently in COMS has a coms-id S3 tag with a value already in use inside COMS, delete the S3 tag and re-generate a new object ID before adding it to COMS.

This resolves a bug with renaming a file and syncing the folder it's in; COMS will mark the file as soft-deleted but fail to pick up the name change (thus resulting in a de-sync between COMS/S3). In BCBox, it will appear as if renaming the file and syncing it removes it.

https://apps.nrs.gov.bc.ca/int/jira/browse/SHOWCASE-3782

Types of changes

Bug fix (non-breaking change which fixes an issue)

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have checked that unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

Motivation

Please see Jira ticket for a more detailed explanation

S3 doesn't support file renaming; any attempt to do so (e.g. in GeoDrive or any external S3 tool/browser) is actually 2 operations:

Create a copy of the existing file with the new name
Delete the existing file

In (1), the new copy will include the same S3 tags and metadata as the original.

In (2), if the bucket has file versioning enabled, then it is a soft deletion. For COMS, this means that the old file still fully exists in its database.

When syncing, COMS will detect (2) as a new file and will try to add it to COMS, using the same coms-id it sees in its S3 tags. Attempting to re-use the same id will result in database errors, since it's the primary key for a COMS object.

Consequences on file version history

Unfortunately, the "new" file with the updated name will lose its version history, since it's held by the "old" copy. However, since it's only soft-deleted in a versioned bucket, the file history technically still exists; it's just that it's not associated with the renamed "active" file.

While it's probably technically possible (on some level) to have COMS "link" the renamed file to the old soft-deleted one, such that you'd get one continuous file history, this is probably overly complex for limited gain - the 2 files are completely separate entities in S3, so you'd have to hack together some novel non-S3 concept to have them linked.

github-actions[bot] commented 3 days ago

Coverage Report

Totals
Statements:	61.18% ( 3060 / 5002 )
Methods:	51.06% ( 336 / 658 )
Lines:	68.26% ( 1841 / 2697 )
Branches:	53.61% ( 883 / 1647 )