Design: Should laminating a file sync its extents?

[tonyhutter]

Currently, UnifyFS requires you to fsync() your extents before laminating the file:

fp = fopen("checkpoint1.chk")
write(fp, <checkpoint data>)
…
fsync(fp)
fclose(fp)
chmod("checkpoint1.chk", 0444)   /* laminate */

Should a laminate implicitly fsync the extents as well? My gut feeling would be yes. Thoughts?

[adammoody]

I keep going back and forth on that.

Implying an fsync on laminate does simplify the calling sequence in the case of a single writer, so it makes sense in that case.

I think apps with multiple writers are still stuck with needing an fsync. The first instinct is to say that all procs can skip the fsync call and just call laminate. However, if all procs call laminate, then there is a race condition so that the first to laminate wins and it locks out the data contributed from the other procs. It's currently up to the application to ensure all data gets in before any process laminates the file with logic like the following:

// flush writes
fsync()
close()

// synchronize across writers to ensure all contributions are committed
MPI_Barrier(comm)

// one rank can now laminate
if (rank == 0) chmod(..., 0444)

Of course, a barrier is not strictly needed, but it's up to the application to ensure that all procs have flushed their data before the file is changed to a laminated state.

We could still imply the fsync to simplify life for single writers, but we'll need to be clear about the above detail in the documentation.

[tonyhutter]

Good points. Ok, so then let's explicitly require fsync(), even for the single writer case, so that people know it has to be done (and to simplify the code).