search

cloud-software-foundation / c5

c5
Apache License 2.0
12 stars 8 forks source link

QuorumDelegatingLog does too much work when opening a preexisting log file #273

Open joshua-g opened 10 years ago

joshua-g commented 10 years ago

Currently, when opening a preexisting log, QuorumDelegatingLog reads the entire log file, including the "contents" (payload) of each entry. A side effect of this is that it will detect a crc error in the written contents; however, maybe this behavior should be configurable?

The simplest improvement is, instead of reading each entry in its entirety, only read the header of each entry. If the header's type is DATA, the content can be skipped. If the type is QUORUM_CONFIGURATION, however, it must be read.

ryanobjc commented 10 years ago

Reading only the headers may not cause less IO to happen. It will reduce the amount of data transfer between kbufcache and user space, so that is a plus.

I guess what I'm trying to say is, we should measure it!!