twmht / python-rocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
276 stars 90 forks source link

Errors when following instructions to build/install python-rocksdb on Ubuntu 16.04 LTS #27

Closed cseberino closed 6 years ago

cseberino commented 6 years ago

Followed all directions but still getting lots of errors below on Ubuntu 16.04 LTS.

Why?

Collecting python-rocksdb Using cached python-rocksdb-0.6.9.tar.gz Requirement already satisfied: setuptools>=25 in ./lib/python2.7/site-packages (from python-rocksdb) (39.0.1) Building wheels for collected packages: python-rocksdb Running setup.py bdist_wheel for python-rocksdb: started Running setup.py bdist_wheel for python-rocksdb: finished with status 'error' Complete output from command /home/cs/Ws/Wb/ee/rocksdb/venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-install-X7kq1z/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-5aKPP5 --python-tag cp27: running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-2.7/rocksdb creating build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests running egg_info writing requirements to python_rocksdb.egg-info/requires.txt writing python_rocksdb.egg-info/PKG-INFO writing top-level names to python_rocksdb.egg-info/top_level.txt writing dependency_links to python_rocksdb.egg-info/dependency_links.txt reading manifest file 'python_rocksdb.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'python_rocksdb.egg-info/SOURCES.txt' copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/options.pxd -> build/lib.linux-x8664-2.7/rocksdb copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb creating build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp running build_ext skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date) building 'rocksdb._rocksdb' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/rocksdb x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ rocksdb/_rocksdb.cpp: In function ‘PyObject __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy, PyObject)’: rocksdb/_rocksdb.cpp:7869:105: warning: conversion to ‘int’ from ‘std::vector::size_type {aka long unsigned int}’ may alter its value [-Wconversion] pyx_t_6->CreateFilter(py_rocks::vector_data(pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst)); ^ rocksdb/_rocksdb.cpp: In function ‘PyObject Pyx_PyInt_From_rocksdb_3a3a_CompressionType(rocksdb::CompressionType)’: rocksdb/_rocksdb.cpp:46834:74: warning: the result of the conversion is unspecified because ‘-1’ is outside the range of type ‘rocksdb::CompressionType’ [-Wconversion] const rocksdb::CompressionType neg_one = (rocksdb::CompressionType) -1, const_zero = (rocksdb::CompressionType) 0; ^ c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -lrocksdb -lsnappy -lbz2 -lz -llz4 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so /usr/bin/ld: cannot find -llz4 collect2: error: ld returned 1 exit status error: command 'c++' failed with exit status 1


Running setup.py clean for python-rocksdb Failed to build python-rocksdb Installing collected packages: python-rocksdb Running setup.py install for python-rocksdb: started Running setup.py install for python-rocksdb: finished with status 'error' Complete output from command /home/cs/Ws/Wb/ee/rocksdb/venv/bin/python2 -u -c "import setuptools, tokenize;file='/tmp/pip-install-X7kq1z/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-vYCQI_/install-record.txt --single-version-externally-managed --compile --install-headers /home/cs/Ws/Wb/ee/rocksdb/venv/include/site/python2.7/python-rocksdb: running install running build running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-2.7/rocksdb creating build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests running egg_info writing requirements to python_rocksdb.egg-info/requires.txt writing python_rocksdb.egg-info/PKG-INFO writing top-level names to python_rocksdb.egg-info/top_level.txt writing dependency_links to python_rocksdb.egg-info/dependency_links.txt reading manifest file 'python_rocksdb.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'python_rocksdb.egg-info/SOURCES.txt' copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/options.pxd -> build/lib.linux-x8664-2.7/rocksdb copying rocksdb/slice.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb creating build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp running build_ext skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date) building 'rocksdb._rocksdb' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/rocksdb x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ rocksdb/_rocksdb.cpp: In function ‘PyObject __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy, PyObject)’: rocksdb/_rocksdb.cpp:7869:105: warning: conversion to ‘int’ from ‘std::vector::size_type {aka long unsigned int}’ may alter its value [-Wconversion] pyx_t_6->CreateFilter(py_rocks::vector_data(pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst)); ^ rocksdb/_rocksdb.cpp: In function ‘PyObject Pyx_PyInt_From_rocksdb_3a3a_CompressionType(rocksdb::CompressionType)’: rocksdb/_rocksdb.cpp:46834:74: warning: the result of the conversion is unspecified because ‘-1’ is outside the range of type ‘rocksdb::CompressionType’ [-Wconversion] const rocksdb::CompressionType neg_one = (rocksdb::CompressionType) -1, const_zero = (rocksdb::CompressionType) 0; ^ c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -lrocksdb -lsnappy -lbz2 -lz -llz4 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so /usr/bin/ld: cannot find -llz4 collect2: error: ld returned 1 exit status error: command 'c++' failed with exit status 1

----------------------------------------
LeonidShamis commented 6 years ago

Same here :(

...
    running build_ext
    skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
    building 'rocksdb._rocksdb' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/rocksdb
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    rocksdb/_rocksdb.cpp: In function ‘PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)’:
    rocksdb/_rocksdb.cpp:7869:105: warning: conversion to ‘int’ from ‘std::vector<rocksdb::Slice>::size_type {aka long unsigned int}’ may alter its value [-Wconversion]
         __pyx_t_6->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
                                                                                                             ^
    rocksdb/_rocksdb.cpp: In function ‘PyObject* __Pyx_PyInt_From_rocksdb_3a__3a_CompressionType(rocksdb::CompressionType)’:
    rocksdb/_rocksdb.cpp:46834:74: warning: the result of the conversion is unspecified because ‘-1’ is outside the range of type ‘rocksdb::CompressionType’ [-Wconversion]
         const rocksdb::CompressionType neg_one = (rocksdb::CompressionType) -1, const_zero = (rocksdb::CompressionType) 0;
                                                                              ^
    c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -lrocksdb -lsnappy -lbz2 -lz -llz4 -o build/lib.linux-x86_64-2.7/rocksdb/_rocksdb.so
    /usr/bin/ld: cannot find -llz4
    collect2: error: ld returned 1 exit status
    error: command 'c++' failed with exit status 1

    ----------------------------------------
Command "/home/leonid/rocksdb/pyrocks_test/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ZdYsTb/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-INlpZq-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/leonid/rocksdb/pyrocks_test/include/site/python2.7/python-rocksdb" failed with error code 1 in /tmp/pip-build-ZdYsTb/python-rocksdb/
(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$
LeonidShamis commented 6 years ago

Solution suggested in https://github.com/twmht/python-rocksdb/issues/22#issuecomment-365946312 has worked for me:

(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$ sudo apt-get install liblz4-dev
...
Setting up liblz4-dev:amd64 (0.0~r131-2ubuntu2) ...
(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$
(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$ pip install python-rocksdb
...
Successfully installed python-rocksdb-0.6.9
(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import rocksdb
>>> db = rocksdb.DB("test.db", rocksdb.Options(create_if_missing=True))
>>> db.put(b'a', b'data')
>>> print db.get(b'a')
data
>>> exit()
(pyrocks_test) leonid@ubuntu:~/rocksdb/pyrocks_test$
twmht commented 6 years ago

closed due to lack of the activity