Congyuwang / RocksDict

Python fast on-disk dictionary / RocksDB & SpeeDB Python binding
https://congyuwang.github.io/RocksDict/rocksdict.html
MIT License
176 stars 8 forks source link

BUG: Exception: utf-8 decoding error (using .with_ttl to write, and any other mode to read) #64

Closed Menziess closed 1 year ago

Menziess commented 1 year ago

Steps

Describe the bug When writing a value to rocksdb using the with_ttl mode, while reading the value using any other mode, a utf-8 decoding error occurs.

To Reproduce Create db using with_ttl, write, close it. Then create the db using read_only, read, error:

from rocksdict import AccessType, Rdict

db = Rdict(
    './test_dict',
    access_type=AccessType.with_ttl(24 * 3600)
)
db[0] = 'test'
db.close()

db = Rdict(
    './test_dict',
    access_type=AccessType.with_ttl(24 * 3600)
)
print(db[0])

db = Rdict(
    './test_dict',
    access_type=AccessType.read_only()
)
print(db[0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: utf-8 decoding error

Expected behavior Expected that the value was read, just as it was using with_ttl mode. Additional info:

➜ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.2 LTS
Release:        22.04
Codename:       jammy
➜ python --version
Python 3.10.6
Congyuwang commented 1 year ago

Hi. This is not a bug. The following is from rocksdb:

Some warnings:

Calling DB::Open directly to re-open a db created by this API will get corrupt values(timestamp suffixed) and no ttl effect will be there during the second Open, so use this API consistently to open the db Be careful when passing ttl with a small positive value because the whole database may be deleted in a small amount of time

Congyuwang commented 1 year ago

It basically says that the time stamp is suffixed to the value in ttl mode. So to correctly read it reopen the database also in ttl mode.

Menziess commented 1 year ago

Okay, for me that was unexpected at least. But that explains it.