Add checksum for ssd cache - Githubissues

facebookincubator / velox

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.

https://velox-lib.io/

Apache License 2.0

3.2k stars 1.06k forks source link

Add checksum for ssd cache #9822

Closed zacw7 closed 1 week ago

zacw7 commented 2 weeks ago

This PR introduces an optional checksum feature for the SSD cache. When enabled, a CRC-based checksum is calculated for each cache entry and stored in the checkpoint file. Additionally, if read verification is activated, the checksum is recalculated and verified against the stored value when cache data is recovered from the checkpoint or loaded from the SSD. A new counter, ssd_cache_read_corruptions, is added to track the number of corruptions detected due to checksum mismatches.

netlify[bot] commented 2 weeks ago

Deploy Preview for meta-velox canceled.

Name	Link
Latest commit	efab2e0c8817527f6370304c7ba7e84ea830c817
Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/665202e5366b1500089b6223

facebook-github-bot commented 2 weeks ago

@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 2 weeks ago

@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 2 weeks ago

@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 2 weeks ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

@zacw7 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

This pull request was exported from Phabricator. Differential Revision: D57394558

facebook-github-bot commented 1 week ago

@zacw7 merged this pull request in facebookincubator/velox@3413ef388711b002f60188ff26bf0b8728a209c2.

conbench-facebook[bot] commented 1 week ago

Conbench analyzed the 1 benchmark run on commit 3413ef38.

There was 1 benchmark result indicating a performance regression:

Commit Run on GitHub-runner-8-core at 2024-05-25 19:53:08Z
- cast_varhar_as_date##try_cast_invalid_input (C++) with source=cpp-micro, suite=velox_cast_benchmark

The full Conbench report has more details.