Provide support for FSBuffer for point lookups. Also added support for scans and compactions that don't go through prefetching.
*Make SstFileWriter create SST files without persisting user defined timestamps when the Option.persist_user_defined_timestamps flag is set to false.
Add support for user-defined timestamps in APIs DeleteFilesInRanges and GetPropertiesOfTablesInRange.
Mark wal_compression feature as production-ready. Currently only compatible with ZSTD compression.
Public API Changes
Allow setting Stderr logger via C API
Declare one Get and one MultiGet variant as pure virtual, and make all the other variants non-overridable. The methods required to be implemented by derived classes of DB allow returning timestamps. It is up to the implementation to check and return an error if timestamps are not supported. The non-batched MultiGet APIs are reimplemented in terms of batched MultiGet, so callers might see a performance improvement.
*Remove the default WritableFile::GetFileSize and FSWritableFile::GetFileSize implementation that returns 0 and make it pure virtual, so that subclasses are enforced to explicitly provide an implementation.
sst_dump --command=check now compares the number of records in a table with num_entries in table property, and reports corruption if there is a mismatch. API SstFileDumper::ReadSequential() is updated to optionally do this verification. (#12322)
Behavior Changes
format_version=6 is the new default setting in BlockBasedTableOptions, for more robust data integrity checking. DBs and SST files written with this setting cannot be read by RocksDB versions before 8.6.0.
Compactions can be scheduled in parallel in an additional scenario: multiple files are marked for compaction within a single column family
For leveled compaction, RocksDB will try to do intra-L0 compaction if the total L0 size is small compared to Lbase (#12214). Users with atomic_flush=true are more likely to see the impact of this change.
Bug Fixes
Fixed a data race in DBImpl::RenameTempFileToOptionsFile.
Fix some perf context statistics error in write steps. which include missing write_memtable_time in unordered_write. missing write_memtable_time in PipelineWrite when Writer stat is STATE_PARALLEL_MEMTABLE_WRITER. missing write_delay_time when calling DelayWrite in WriteImplWALOnly function.
Fixed a bug that can, under rare circumstances, cause MultiGet to return an incorrect result for a duplicate key in a MultiGet batch.
Fix a bug where older data of an ingested key can be returned for read when universal compaction is used
Provide support for FSBuffer for point lookups. Also added support for scans and compactions that don't go through prefetching.
*Make SstFileWriter create SST files without persisting user defined timestamps when the Option.persist_user_defined_timestamps flag is set to false.
Add support for user-defined timestamps in APIs DeleteFilesInRanges and GetPropertiesOfTablesInRange.
Mark wal_compression feature as production-ready. Currently only compatible with ZSTD compression.
Public API Changes
Allow setting Stderr logger via C API
Declare one Get and one MultiGet variant as pure virtual, and make all the other variants non-overridable. The methods required to be implemented by derived classes of DB allow returning timestamps. It is up to the implementation to check and return an error if timestamps are not supported. The non-batched MultiGet APIs are reimplemented in terms of batched MultiGet, so callers might see a performance improvement.
*Remove the default WritableFile::GetFileSize and FSWritableFile::GetFileSize implementation that returns 0 and make it pure virtual, so that subclasses are enforced to explicitly provide an implementation.
sst_dump --command=check now compares the number of records in a table with num_entries in table property, and reports corruption if there is a mismatch. API SstFileDumper::ReadSequential() is updated to optionally do this verification. (#12322)
Behavior Changes
format_version=6 is the new default setting in BlockBasedTableOptions, for more robust data integrity checking. DBs and SST files written with this setting cannot be read by RocksDB versions before 8.6.0.
Compactions can be scheduled in parallel in an additional scenario: multiple files are marked for compaction within a single column family
For leveled compaction, RocksDB will try to do intra-L0 compaction if the total L0 size is small compared to Lbase (#12214). Users with atomic_flush=true are more likely to see the impact of this change.
Bug Fixes
Fixed a data race in DBImpl::RenameTempFileToOptionsFile.
Fix some perf context statistics error in write steps. which include missing write_memtable_time in unordered_write. missing write_memtable_time in PipelineWrite when Writer stat is STATE_PARALLEL_MEMTABLE_WRITER. missing write_delay_time when calling DelayWrite in WriteImplWALOnly function.
Fixed a bug that can, under rare circumstances, cause MultiGet to return an incorrect result for a duplicate key in a MultiGet batch.
Fix a bug where older data of an ingested key can be returned for read when universal compaction is used
8.11.0 (01/19/2024)
New Features
Add new statistics: rocksdb.sst.write.micros measures time of each write to SST file; rocksdb.file.write.{flush|compaction|db.open}.micros measure time of each write to SST table (currently only block-based table format) and blob file for flush, compaction and db open.
Public API Changes
Added another enumerator kVerify to enum class FileOperationType in listener.h. Update your switch statements as needed.
Add CompressionOptions to the CompressedSecondaryCacheOptions structure to allow users to specify library specific options when creating the compressed secondary cache.
Deprecated several options: level_compaction_dynamic_file_size, ignore_max_compaction_bytes_for_input, check_flush_compaction_key_order, flush_verify_memtable_count, compaction_verify_record_count, fail_if_options_file_error, and enforce_single_del_contracts
Exposed options ttl via c api.
Behavior Changes
rocksdb.blobdb.blob.file.write.micros expands to also measure time writing the header and footer. Therefore the COUNT may be higher and values may be smaller than before. For stacked BlobDB, it no longer measures the time of explictly flushing blob file.
Files will be compacted to the next level if the data age exceeds periodic_compaction_seconds except for the last level.
Reduced the compaction debt ratio trigger for scheduling parallel compactions
For leveled compaction with default compaction pri (kMinOverlappingRatio), files marked for compaction will be prioritized over files not marked when picking a file from a level for compaction.
Bug Fixes
Fix bug in auto_readahead_size that combined with IndexType::kBinarySearchWithFirstKey + fails or iterator lands at a wrong key
Fixed some cases in which DB file corruption was detected but ignored on creating a backup with BackupEngine.
... (truncated)
Commits
f444196 Add a FS flag to detect and correct corruption (#12408)
9ded0f7 Fix regression for Javadoc jar build (#12404)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Bumps org.rocksdb:rocksdbjni from 8.11.3 to 9.0.0.
Release notes
Sourced from org.rocksdb:rocksdbjni's releases.
Changelog
Sourced from org.rocksdb:rocksdbjni's changelog.
... (truncated)
Commits
f444196
Add a FS flag to detect and correct corruption (#12408)9ded0f7
Fix regression for Javadoc jar build (#12404)b64bc23
Fix windows build and CI (#12426)3c9956e
Correct CMake Javadoc and source jar builds (#12371)659fc7c
Update HISTORY.md for 9.0.0f2732d0
Export GetSequenceNumber functionality for Snapshots (#12354)055b21a
Update ZLib to 1.3.1 (#12358)d227276
Deprecate some variants of Get and MultiGet (#12327)956f1df
Change ReadAsync callback API to remove const from FSReadRequest (#11649)28c1c15
Sync tickers and histograms across C++ and Java (#12355)Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show