search

hpc / mpifileutils

File utilities designed for scalability and performance.
https://hpc.github.io/mpifileutils
BSD 3-Clause "New" or "Revised" License
162 stars 64 forks source link

Add O_NOATIME to the flags for file open so the source file's atime i… #534

Closed dpjohnson closed 9 months ago

dpjohnson commented 1 year ago

…s preserved (if the platform supports it). Signed-off-by: Doug Johnson djohnson@osc.edu.

Lightly tested dsync on Linux (Red Hat 7.9). Not sure if this covers the bases for what's needed to reliably not change the source file's atime. I went off a comment src/common/mfu_flist.h:29 where the _GNU_SOURCE define appears to have been intended to be used for this purpose. I checked the open and closed issues and pull requests to see if this was ever discussed, surprisingly absent. This feature would be useful for those migrating data and that use atime for purging or migration purposes: the source file's atime needs to be preserved in the interregnum between when the copy is initiated, subsequent invocations of dsync, and the cutover to the new location of the data is performed.

adammoody commented 9 months ago

@dpjohnson , we'd like to make stamp a new release of mpiFileUtils in the next month or two.

I have a PR to add O_NOATIME here: https://github.com/hpc/mpifileutils/pull/561

Thanks for making this suggestion.

adammoody commented 9 months ago

@dpjohnson , I just merged the PR to add support for O_NOATIME. Several tools, including dsync now offer a --open-noatime option to enable this. If you find problems with that, please let me know.

Sorry for the slow turn around on this. But thanks again for your suggestion and test PR!