twmht
commented
5 years ago @PlatonB
the version of rocksdb you used?
Open PlatonB opened 5 years ago
twmht
commented
5 years ago @PlatonB
the version of rocksdb you used?
marcosschroh
commented
5 years ago Hi,
I can not installed as well.
python version: 3.7
Running setup.py install for python-rocksdb: started
Running setup.py install for python-rocksdb: finished with status 'error'
Complete output from command /usr/local/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-xst7wcfu/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-hlbvsws0/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/interfaces.py -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/__init__.py -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/errors.py -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-3.7/rocksdb
creating build/lib.linux-x86_64-3.7/rocksdb/tests
copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-3.7/rocksdb/tests
copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.7/rocksdb/tests
copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-3.7/rocksdb/tests
copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-3.7/rocksdb/tests
running egg_info
writing python_rocksdb.egg-info/PKG-INFO
writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
writing requirements to python_rocksdb.egg-info/requires.txt
writing top-level names to python_rocksdb.egg-info/top_level.txt
reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/cache.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/env.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/options.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/status.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-3.7/rocksdb
copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-3.7/rocksdb
creating build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-3.7/rocksdb/cpp
running build_ext
skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
building 'rocksdb._rocksdb' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/rocksdb
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.7m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_16compression_opts___get__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*)’:
rocksdb/_rocksdb.cpp:17948:79: error: ‘struct rocksdb::CompressionOptions’ has no member named ‘max_dict_bytes’
__pyx_t_1 = __Pyx_PyInt_From_uint32_t(__pyx_v_self->copts->compression_opts.max_dict_bytes); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 893, __pyx_L1_error)
^~~~~~~~~~~~~~
rocksdb/_rocksdb.cpp: In function ‘int __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_16compression_opts_2__set__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*, PyObject*)’:
rocksdb/_rocksdb.cpp:18184:20: error: ‘struct rocksdb::CompressionOptions’ has no member named ‘max_dict_bytes’
__pyx_v_copts->max_dict_bytes = __pyx_t_5;
^~~~~~~~~~~~~~
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_14compaction_pri___get__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*)’:
rocksdb/_rocksdb.cpp:18349:56: error: ‘kMinOverlappingRatio’ is not a member of ‘rocksdb’
__pyx_t_1 = ((__pyx_v_self->copts->compaction_pri == rocksdb::kMinOverlappingRatio) != 0);
^~~~~~~
rocksdb/_rocksdb.cpp: In function ‘int __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_14compaction_pri_2__set__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*, PyObject*)’:
rocksdb/_rocksdb.cpp:18549:43: error: ‘kMinOverlappingRatio’ is not a member of ‘rocksdb’
__pyx_v_self->copts->compaction_pri = rocksdb::kMinOverlappingRatio;
^~~~~~~
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_11compression___get__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*)’:
rocksdb/_rocksdb.cpp:18781:10: error: ‘kXpressCompression’ is not a member of ‘rocksdb’
case rocksdb::kXpressCompression:
^~~~~~~
rocksdb/_rocksdb.cpp:18805:10: error: ‘kZSTD’ is not a member of ‘rocksdb’
case rocksdb::kZSTD:
^~~~~~~
rocksdb/_rocksdb.cpp:18853:10: error: ‘kDisableCompressionOption’ is not a member of ‘rocksdb’
case rocksdb::kDisableCompressionOption:
^~~~~~~
rocksdb/_rocksdb.cpp: In function ‘int __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_11compression_2__set__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*, PyObject*)’:
rocksdb/_rocksdb.cpp:19178:40: error: ‘kZSTD’ is not a member of ‘rocksdb’
__pyx_v_self->copts->compression = rocksdb::kZSTD;
^~~~~~~
rocksdb/_rocksdb.cpp:19246:40: error: ‘kDisableCompressionOption’ is not a member of ‘rocksdb’
__pyx_v_self->copts->compression = rocksdb::kDisableCompressionOption;
^~~~~~~
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_20max_compaction_bytes___get__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*)’:
rocksdb/_rocksdb.cpp:19333:62: error: ‘struct rocksdb::ColumnFamilyOptions’ has no member named ‘max_compaction_bytes’; did you mean ‘compaction_filter’?
__pyx_t_1 = __Pyx_PyInt_From_uint64_t(__pyx_v_self->copts->max_compaction_bytes); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 981, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~
rocksdb/_rocksdb.cpp: In function ‘int __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_20max_compaction_bytes_2__set__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*, PyObject*)’:
rocksdb/_rocksdb.cpp:19393:24: error: ‘struct rocksdb::ColumnFamilyOptions’ has no member named ‘max_compaction_bytes’; did you mean ‘compaction_filter’?
__pyx_v_self->copts->max_compaction_bytes = __pyx_t_1;
^~~~~~~~~~~~~~~~~~~~
rocksdb/_rocksdb.cpp: In function ‘int __pyx_pf_7rocksdb_8_rocksdb_19ColumnFamilyOptions_30max_bytes_for_level_multiplier_2__set__(__pyx_obj_7rocksdb_8_rocksdb_ColumnFamilyOptions*, PyObject*)’:
rocksdb/_rocksdb.cpp:20437:57: warning: conversion to ‘int’ from ‘double’ may alter its value [-Wfloat-conversion]
__pyx_v_self->copts->max_bytes_for_level_multiplier = __pyx_t_1;
^~~~~~~~~
rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_f_7rocksdb_8_rocksdb_12BaseIterator_seek_for_prev(__pyx_obj_7rocksdb_8_rocksdb_BaseIterator*, PyObject*, int)’:
rocksdb/_rocksdb.cpp:40161:30: error: ‘class rocksdb::Iterator’ has no member named ‘SeekForPrev’; did you mean ‘SeekToFirst’?
__pyx_v_self->ptr->SeekForPrev(__pyx_v_c_key);
^~~~~~~~~~~
error: command 'gcc' failed with exit status 1Any ideas?
iFA88
commented
5 years ago This is similar for me when i have git cloned the latest rocksdb branch. I suggest to try this: https://github.com/facebook/rocksdb/archive/v5.18.3.tar.gz extract, then:
cd rocksdb-5.18.3
mkdir build && cd build
cmake ..
make
cd ..
export CPLUS_INCLUDE_PATH=${CPLUS_INCLUDE_PATH}${CPLUS_INCLUDE_PATH:+:}`pwd`/include/
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}${LD_LIBRARY_PATH:+:}`pwd`/build/
export LIBRARY_PATH=${LIBRARY_PATH}${LIBRARY_PATH:+:}`pwd`/build/Hi @iFA88
Thanks for the answer. So, according to what you are saying I have to install rocksdb first and then python-rocksdb? I have rocksdb running in a container and I have separate container with python code that has to talk with the database. How would you approach this scenario?
iFA88
commented
5 years ago You need first install rocksdb library, set environment with export that the pip compiler knows where are the headers and then you can install python-rocksdb with pip.
If you use virtualenv you need use source before you launch the pip installer.
TomGoBravo
commented
5 years ago @marcosschroh python-rocksdb links the rocksdb cpp code; they need to be installed in the same container. @PlatonB : Did you manage to install the rocksdb library and headers? @twmht Without more details from the original poster I think it makes sense to close this issue.
marcosschroh
commented
5 years ago @TomGoBravo yes, I have managed to install rocksdb. Works perfectly with faust
jkgenser
commented
3 years ago @iFA88 : Do you happen to have an example dockerfile of building your fork of rocksdb successfully? I've been trying to build using facebook but I keep getting an error with linking at the very end of the build.
iFA88
commented
3 years ago @iFA88 : Do you happen to have an example dockerfile of building your fork of rocksdb successfully? I've been trying to build using facebook but I keep getting an error with linking at the very end of the build.
I did not have any, but paste your docker builder file content here, let me see.
hyerim-kim
commented
3 years ago @jkgenser I have been resolved this issue without build rocsdb using under command.
RUN apt update && apt install -y python3-pip \
libgflags-dev \
libsnappy-dev \
zlib1g-dev \
libbz2-dev \
liblz4-dev \
libzstd-dev \
librocksdb-devAnd also I have been succeeded build rocksdb using dockerfile and installed python-rocksdb after resolving the above errors, but I'm not sure it helps you. Here is a part of my dockerfile to build and install rocksdb. I wish it can help you.
RUN apt update && apt install -y python3-pip \
build-essential \
cmake \
libgflags-dev \
libsnappy-dev \
zlib1g-dev \
libbz2-dev \
liblz4-dev \
libzstd-dev
RUN git clone https://github.com/facebook/rocksdb.git
WORKDIR /rocksdb
RUN mkdir build
WORKDIR /rocksdb/build
RUN cmake ..
RUN make
RUN make install INSTALL_PATH=/usr
RUN pip install -r requirements.txt
Gabriel-Gardin
commented
3 years ago I get the same error when I install it in my default python(not virtualenv) but I can install it with no problems inside venv
Gabriel-Gardin
commented
3 years ago Turns out I didn't had Cython installed in my local python, after install it I managed to also install python-rocksdb
Congyuwang
commented
2 years ago Try this: https://github.com/Congyuwang/RocksDict.
pip install rocksdict
No need to compile. Pre-built wheels for python 3.6-3.10, Windows, Mac, and Linux
elementary OS 5.0 Python 3.6.7