tikv / raft-engine

A persistent storage engine for Multi-Raft log
Apache License 2.0
565 stars 88 forks source link

Interrupted write causes corruption #243

Closed tabokie closed 2 years ago

tabokie commented 2 years ago

At this line, we use write_all to append some bytes: https://github.com/tikv/raft-engine/blob/ee0f6cf0d9fd5c4839463bd84edbc009402dffda/src/file_pipe_log/log_file.rs#L107

If this write is interrupted, we directly bubble its error. But some portion of the data might already be written. In this case, the self.written is inconsistent with underlying writer's internal offset. A fractured write will remain as a phantom record.

LykxSassinator commented 2 years ago

Maybe we need a new func to support a safe write here? Just like:

index 3f7628c..5f37581 100644
--- a/src/env/mod.rs
+++ b/src/env/mod.rs
@@ -55,4 +55,8 @@ pub trait WriteExt {
     fn truncate(&mut self, offset: usize) -> Result<()>;
     fn sync(&mut self) -> Result<()>;
     fn allocate(&mut self, offset: usize, size: usize) -> Result<()>;
+    fn write_all_safely(
+        &mut self,
+        buf: &mut [u8],
+    ) -> ::std::result::Result<usize, (usize, std::io::Error)>;
 }
tabokie commented 2 years ago

No, simply reseek the writer if there's a failure. If that seek fails, panic.

LykxSassinator commented 2 years ago

Emm...I didn't get it. reseek is just a op which resets the offset, and in our self-defined write, reseek and its followed operations in write is just like a redo op by continue triggered by Errno: EINTER.

https://github.com/LykxSassinator/raft-engine/blob/f3c268bb954f63f2809f84a940141db4419b2c44/src/env/default.rs#L116-L138

And I just wanna introduce a safe write by write_all_safely. If it failed, it would return the tuple, both containing the actual written size of bytes and the error details.

tabokie commented 2 years ago

The purpose here is to make sure subsequent writes can correctly overwrite the failed partial write, so that LogFileWriter::written is always consistent with LogFile::offset, no phantom data is inserted.