No matter what, we need to do a stat call on each file to get the file's size. This stat call also gives us all the file's metadata (permissions, UID, GID, creation times) "for free". This patch encodes the size and all metadata by default into the state_file. This is needed for finalizing transfers after a burst buffer transfer has finished, and the source files are no longer available.
This change does not affect AXL_COPY_METADATA. It must still be set in order to apply the metadata to the file after it has been transferred. This patch does however, do an additional file size check at the end of a transfer, at a cost of a stat per file.
adammoody
commented
3 years ago
No matter what, we need to do a stat call on each file to get the file's size. This stat call also gives us all the file's metadata (permissions, UID, GID, creation times) "for free". This patch encodes the size and all metadata by default into the state_file. This is needed for finalizing transfers after a burst buffer transfer has finished, and the source files are no longer available.
This change does not affect
AXL_COPY_METADATA. It must still be set in order to apply the metadata to the file after it has been transferred. This patch does however, do an additional file size check at the end of a transfer, at a cost of a stat per file.