search

NVSL / linux-nova

NOVA is a log-structured file system designed for byte-addressable non-volatile memories, developed at the University of California, San Diego.
http://nvsl.ucsd.edu/index.php?path=projects/nova
Other
421 stars 118 forks source link

Why does NOVA perform CoW for append write? #73

Open leftgeek opened 5 years ago

leftgeek commented 5 years ago

It seems that NOVA also perform CoW for append write, is this necessary?

Andiry commented 5 years ago

It does not make any difference. Append write is always writing to new blocks.

Thanks, Andiry

On Wed, Jul 17, 2019 at 9:07 AM kongnull notifications@github.com wrote:

It seems that NOVA also perform CoW for append write, is this necessary?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NVSL/linux-nova/issues/73?email_source=notifications&email_token=AAKBYEHPMGIB4KHN2COVNYDP747SDA5CNFSM4IESBJV2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G7ZDBQA, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKBYEH6Y6CZ3JZHOIENZC3P747SDANCNFSM4IESBJVQ .

leftgeek commented 5 years ago

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

stevenjswanson commented 5 years ago

COW should not be necessary for appends. We have been implemented this for some of our recent work, but I’m not sure we released that version. Juno, have we pushed the code for atomic append w/o COW?

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 16:44, kongnull notifications@github.com wrote:

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

juno-kim commented 5 years ago

As far as I know, it is not released yet.

Thanks, Juno

On Wed, Jul 17, 2019 at 8:02 PM Steven Swanson notifications@github.com wrote:

COW should not be necessary for appends. We have been implemented this for some of our recent work, but I’m not sure we released that version. Juno, have we pushed the code for atomic append w/o COW?

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 16:44, kongnull notifications@github.com wrote:

I mean... If the original file size is not 4KB-aligned, append write can simply overwrite the tail block instead of writing to a new block and copying the original data. Is this right?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NVSL/linux-nova/issues/73?email_source=notifications&email_token=AALTYUI5RHNMONE6IU7JBXTP77MNFA5CNFSM4IESBJV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2HFYWI#issuecomment-512646233, or mute the thread https://github.com/notifications/unsubscribe-auth/AALTYUOEKHCDNMCQXX3RAGDP77MNFANCNFSM4IESBJVQ .

leftgeek commented 5 years ago

Thanks, another similar question is that NOVA memsets the rest of the last, partially-written block (the portion beyonds file size) to zero (and flushes it). This seems also unnecessary?

stevenjswanson commented 5 years ago

This is a security issue. If you mmap the file you will get access to the unwritten space. If we don’t zero it, it could contain old data from a previously deleted file.

It also might be a POSIX compatibly issue. I would guess that areas is supposed to be zeroed for mmap.

-steve

-- Composed on (and maybe dictated to) my phone.

On Jul 17, 2019, at 20:26, kongnull notifications@github.com wrote:

Thanks, another similar question is that NOVA memsets the rest of the last, partially-written block (the portion beyonds file size) to zero (and flushes it). This seems also unnecessary?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

leftgeek commented 5 years ago

Thanks a lot. You're right, I found an explanation at http://www.man7.org/linux/man-pages/man2/mmap.2.html

A file is mapped in multiples of the page size. For a file that is not a multiple of the page size, the remaining memory is zeroed when mapped, and writes to that region are not written out to the file.

I guess maybe we can delay the memset operation until a mmap() call?