twmht / python-rocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
274 stars 89 forks source link

Can't pip install in windows #98

Open fonty422 opened 3 years ago

fonty422 commented 3 years ago

I'm trying to use Faust, but it fails to correctly set up the rocksdb. Unless I'm missing something obvious, here's the link to the issue I raised for Faust.

We tried to download the source too and then run the setup.py directly, even tried removing the checks for Wextra and Wconversion (the two that seem to fail) and it then fails for other reasons.

I don't have Anaconda installed (this prove to be problematic with previously installed python and the jupyter notebook) so can't use that either. Any assistance would be greatly appreciated.

Congyuwang commented 2 years ago

Try this. This is also implemented in RocksDB. A python on-disk storage. Great ergonomics.

https://github.com/Congyuwang/RocksDict

Store any python object on-disk also.

You don't need to compile. Just pip install rocksdict. It is pre-compiled.

from rocksdict import Rdict
import numpy as np
import pandas as pd

path = str("./test_dict")

# create a Rdict with default options at `path`
db = Rdict(path)

db[1.0] = 1
db[1] = 1.0
db["huge integer"] = 2343546543243564534233536434567543
db["good"] = True
db["bad"] = False
db["bytes"] = b"bytes"
db["this is a list"] = [1, 2, 3]
db["store a dict"] = {0: 1}
db[b"numpy"] = np.array([1, 2, 3])
db["a table"] = pd.DataFrame({"a": [1, 2], "b": [2, 1]})

# close Rdict
db.close()

# reopen Rdict from disk
db = Rdict(path)
assert db[1.0] == 1
assert db[1] == 1.0
assert db["huge integer"] == 2343546543243564534233536434567543
assert db["good"] == True
assert db["bad"] == False
assert db["bytes"] == b"bytes"
assert db["this is a list"] == [1, 2, 3]
assert db["store a dict"] == {0: 1}
assert np.all(db[b"numpy"] == np.array([1, 2, 3]))
assert np.all(db["a table"] == pd.DataFrame({"a": [1, 2], "b": [2, 1]}))

# iterate through all elements
for k, v in db.items():
    print(f"{k} -> {v}")

# batch get:
print(db[["good", "bad", 1.0]])
# [True, False, 1]

# delete Rdict from dict
db.close()
Rdict.destroy(path)