search

blugelabs / bluge

indexing library for Go
Apache License 2.0
1.9k stars 124 forks source link

log message "error loading snapshot" reported under simple workload #40

Open mschoch opened 3 years ago

mschoch commented 3 years ago

@rubiojr reports here https://github.com/blugelabs/bluge/pull/39 that when using a Reader not obtained from the Writer and while the Writer is still open, the message "error loading snapshot epoch: %d: %v" appears.

mschoch commented 3 years ago

One theory is that as the Writer independently creates new snapshots, when opening a Reader (not via the Writer) it may attempt to open a partially written snapshot file. This would fail the CRC check, and another snapshot is tried.

We should be able to test this theory with a straightforward unit test.

One approach to fix this might be to write new snapshot files with a different extension, and rename them after a complete snapshot has been written. This would prevent new readers from seeing partially written snapshot files.