search

twmht / python-rocksdb

Python bindings for RocksDB
BSD 3-Clause "New" or "Revised" License
274 stars 89 forks source link

Unable To get "pip3 install python-rocksdb" working #84

Open akul-goyal opened 3 years ago

akul-goyal commented 3 years ago

Hi,

I have been trying to get python-rocksdb to run on my vagrant Ubuntu 18.04. I have tried various solutions online and they have not worked. So far the best method has been: https://pyrocksdb.readthedocs.io/en/v0.4/installation.html

I used that to install rocksdb. However trying to get python-rocksdb to work through: pip3 install git+git://github.com/stephan-hof/pyrocksdb.git or pip3 install python-rocksdb does not work. I get the error pasted below when running: pip install python-rocksdb

I also have Cpython installed too.

Any help would be awesome!

 vagrant@vagrant:~/rocksdb$ pip3 install python-rocksdb
Collecting python-rocksdb
  Downloading https://files.pythonhosted.org/packages/94/81/9a8e498a26ec34da9e1ae87e831468aedb4a6ba42aa9ea779b97896cfea7/python-rocksdb-0.7.0.tar.gz (219kB)
    100% |████████████████████████████████| 225kB 4.4MB/s 
Collecting setuptools>=25 (from python-rocksdb)
  Using cached https://files.pythonhosted.org/packages/6d/38/c21ef5034684ffc0412deefbb07d66678332290c14bb5269c85145fbd55e/setuptools-50.3.2-py3-none-any.whl
Building wheels for collected packages: python-rocksdb
  Running setup.py bdist_wheel for python-rocksdb ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-cg1fvohs/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp97o9zad5pip-wheel- --python-tag cp36:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/interfaces.py -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/errors.py -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb
  creating build/lib.linux-x86_64-3.6/rocksdb/tests
  copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
  copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
  copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
  copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
  running egg_info
  writing python_rocksdb.egg-info/PKG-INFO
  writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
  writing requirements to python_rocksdb.egg-info/requires.txt
  writing top-level names to python_rocksdb.egg-info/top_level.txt
  reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
  copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/cache.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/env.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/options.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/status.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-3.6/rocksdb
  creating build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
  running build_ext
  cythoning rocksdb/_rocksdb.pyx to rocksdb/_rocksdb.cpp
  /tmp/pip-build-cg1fvohs/python-rocksdb/.eggs/Cython-0.29.21-py3.6-linux-x86_64.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-build-cg1fvohs/python-rocksdb/rocksdb/_rocksdb.pyx
    tree = Parsing.p_module(s, pxd, full_module_name)
  building 'rocksdb._rocksdb' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/rocksdb
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.6/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
  rocksdb/_rocksdb.cpp:634:10: fatal error: rocksdb/slice.h: No such file or directory
   #include "rocksdb/slice.h"
            ^~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for python-rocksdb
  Running setup.py clean for python-rocksdb
Failed to build python-rocksdb
Installing collected packages: setuptools, python-rocksdb
  Running setup.py install for python-rocksdb ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-cg1fvohs/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-1hw2m1f6-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/interfaces.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/merge_operators.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/errors.py -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb
    creating build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/__init__.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    copying rocksdb/tests/test_memtable.py -> build/lib.linux-x86_64-3.6/rocksdb/tests
    running egg_info
    writing python_rocksdb.egg-info/PKG-INFO
    writing dependency_links to python_rocksdb.egg-info/dependency_links.txt
    writing requirements to python_rocksdb.egg-info/requires.txt
    writing top-level names to python_rocksdb.egg-info/top_level.txt
    reading manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'python_rocksdb.egg-info/SOURCES.txt'
    copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/backup.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/cache.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/db.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/env.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/logger.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/options.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/status.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-3.6/rocksdb
    creating build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-3.6/rocksdb/cpp
    running build_ext
    skipping 'rocksdb/_rocksdb.cpp' Cython extension (up-to-date)
    building 'rocksdb._rocksdb' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/rocksdb
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-3.6/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing -fno-rtti
    rocksdb/_rocksdb.cpp:634:10: fatal error: rocksdb/slice.h: No such file or directory
     #include "rocksdb/slice.h"
              ^~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-cg1fvohs/python-rocksdb/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-1hw2m1f6-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-cg1fvohs/python-rocksdb/
ylogx commented 3 years ago

Duplicate of #51

Jamiejoin commented 3 years ago

Or you can consider using the rocksdb environment with docker to mount the code and data