wtsi-hgi / hgi-vault

Data retention policy tools
GNU General Public License v3.0
1 stars 0 forks source link

Judge file age using the greater of atime, mtime & ctime #21

Closed sb10 closed 2 years ago

sb10 commented 2 years ago

File age (and thus eligibility for deletion) is based on comparing the current time to the mtime of the file.

This was done because:

We use modification time, rather than change time, as it's a better indicator of usage. (Unfortunately, access time is not reliably available to us on all filesystems.)

However, there are scenarios where we can have unexpected, undesirable deletion of files without warning, due to the mtime of "new" files being too old:

Using the greater of atime, mtime & ctime will be no worse than the current implementation, but likely better in these scenarios.

sb10 commented 2 years ago

Judge file age using the greater of atime, mtime & ctime

sb10 commented 2 years ago

Actually, don't consider ctime, since we expect sweeping permission changes, and don't want this to make all old files look active. We also can't change ctime during tests.

When you untar an archive, you don't get the original ctimes, so this shouldn't affect the desired functionality.

sb10 commented 2 years ago

After discussion in https://github.com/wtsi-hgi/hgi-vault/pull/27 which attempted to resolve this issue, it's better to consider ctime as well.

In summary, when permissions are changed, it is most likely happening on a new file anyway, and by checking ctime we err on the side of caution and never delete newly added files, no matter how they were put on the disk.

sb10 commented 2 years ago

As a user when I add a file to a project it shouldn't be deleted without warning