couchbase / moss

moss - a simple, fast, ordered, persistable, key-val storage library for golang
Other
959 stars 58 forks source link

is a large pwrite() more efficient than many concurrent smaller pwrite()'s? #32

Open steveyen opened 7 years ago

steveyen commented 7 years ago

In moss, it might have a large segment (many megabytes) that needs to be written or appended to the end of the file.

Is it more efficient to call pwrite() once with the entire segment's large data buffer, where perhaps the OS/filesystem knows how to efficiently split that up into concurrent I/O channel use...

...or, should moss split the big buffer into multiple, smaller, block-aligned sections, where moss can spawn off multiple goroutines to invoke several pwrite()'s concurrently.

Which one is faster / utilizes more I/O bandwidth or channels?