search

c-w / gutenberg

A simple interface to the Project Gutenberg corpus.
Apache License 2.0
321 stars 59 forks source link

pip install gutenberg failed in the anaconda/conda environment #133

Closed mikechen66 closed 4 years ago

mikechen66 commented 4 years ago

Dear Sir:

I tried many times but I failed the installation in the Python 3.7 environment on Anaconda/Conda Environment (My original Python is Pythonb 3.6). I also tried Berkeley DB 5.3 but also failed. Please give me a help. There are three scenarios listed for your reference.

Scenario One:

(base) mike@mc $ pip install gutenberg

It reminded me the following error message. 

 ERROR: Command errored out with exit status 1:
   command: /home/mike/miniconda3/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-7gv98mt0
       cwd: /tmp/pip-install-j0827hki/bsddb3/

Please look at the following long detailed information if necessary.

Scenario Two:

While I tried to install gutenberg with the sudo command. Gutenberg was installed in the original Python 3.6 environment. But I could not use it in the Anaconda/conda environment.

(base) mike@mc $ sudo pip install gutenberg 

Successfully build gutenberg   # install Gutenberg on the original Python 3.6 environment 
Installing collected packages: gutenberg
Successfully installed gutenberg-0.8.1.

(base) mike@mc $ python
Python 3.7.4 (default, Aug 13 2019, 20:35:49) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gutenberg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gutenberg'
>>> from gutenberg.acquire import load_etext
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gutenberg'

During the uninstallation of Gutenberg, I find its path is listed as follows. 
(base) mike@mc $ sudo pip uninstall gutenberg
.......
/usr/local/lib/python3.6/dist-packages/Gutenberg-0.8.1.dist-info/*
/usr/local/lib/python3.6/dist-packages/Gutenberg/*

Scenario Three: 

(base) mike@mc:~$ sudo apt-get install libdb++-dev
[sudo] password for mike: 
Sorry, try again.
[sudo] password for mike: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libdb++-dev is already the newest version (1:5.3.21~exp1ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 291 not upgraded.
(base) mike@mc:~$ export BERKELEYDB_DIR=/usr
(base) mike@mc:~$ pip install .
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

Note:

Please see the following first scenario in details.

(base) mike@mc:~$ pip install gutenberg
Processing ./.cache/pip/wheels/cb/c2/c3/17d6ce99a54f6f850047cb089c7d649d6af8048bc3f186b019/Gutenberg-0.8.1-py3-none-any.whl
Requirement already satisfied: six>=1.10.0 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (1.14.0)
Requirement already satisfied: future>=0.15.2 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (0.18.2)
Requirement already satisfied: SPARQLWrapper>=1.8.2 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (1.8.5)
Requirement already satisfied: rdflib<5.0.0,>=4.2.0 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (4.2.2)
Collecting bsddb3>=6.1.0
  Using cached bsddb3-6.2.7.tar.gz (437 kB)
Requirement already satisfied: requests>=2.5.1 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (2.23.0)
Requirement already satisfied: setuptools>=18.5 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (46.1.3.post20200330)
Requirement already satisfied: rdflib-sqlalchemy>=0.3.8 in ./miniconda3/lib/python3.7/site-packages (from gutenberg) (0.4.0)
Requirement already satisfied: pyparsing in ./miniconda3/lib/python3.7/site-packages (from rdflib<5.0.0,>=4.2.0->gutenberg) (2.4.6)
Requirement already satisfied: isodate in ./miniconda3/lib/python3.7/site-packages (from rdflib<5.0.0,>=4.2.0->gutenberg) (0.6.0)
Requirement already satisfied: idna<3,>=2.5 in ./miniconda3/lib/python3.7/site-packages (from requests>=2.5.1->gutenberg) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in ./miniconda3/lib/python3.7/site-packages (from requests>=2.5.1->gutenberg) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./miniconda3/lib/python3.7/site-packages (from requests>=2.5.1->gutenberg) (1.25.8)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.7/site-packages (from requests>=2.5.1->gutenberg) (2020.4.5.1)
Requirement already satisfied: alembic>=0.8.8 in ./miniconda3/lib/python3.7/site-packages (from rdflib-sqlalchemy>=0.3.8->gutenberg) (1.4.2)
Requirement already satisfied: SQLAlchemy>=1.1.4 in ./miniconda3/lib/python3.7/site-packages (from rdflib-sqlalchemy>=0.3.8->gutenberg) (1.3.16)
Requirement already satisfied: python-dateutil in ./miniconda3/lib/python3.7/site-packages (from alembic>=0.8.8->rdflib-sqlalchemy>=0.3.8->gutenberg) (2.8.1)
Requirement already satisfied: python-editor>=0.3 in ./miniconda3/lib/python3.7/site-packages (from alembic>=0.8.8->rdflib-sqlalchemy>=0.3.8->gutenberg) (1.0.4)
Requirement already satisfied: Mako in ./miniconda3/lib/python3.7/site-packages (from alembic>=0.8.8->rdflib-sqlalchemy>=0.3.8->gutenberg) (1.1.2)
Requirement already satisfied: MarkupSafe>=0.9.2 in ./miniconda3/lib/python3.7/site-packages (from Mako->alembic>=0.8.8->rdflib-sqlalchemy>=0.3.8->gutenberg) (1.1.1)
Building wheels for collected packages: bsddb3
  Building wheel for bsddb3 (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/mike/miniconda3/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-7gv98mt0
       cwd: /tmp/pip-install-j0827hki/bsddb3/
  Complete output (56 lines):
  Found Berkeley DB 5.3 installation.
    include files in /usr/include
    library files in /usr/lib/x86_64-linux-gnu
    library name is libdb-5.3
  Detected Berkeley DB version 5.3 from db.h
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/dbutils.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/dbrecio.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/dbshelve.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/dbtables.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/dbobj.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/__init__.py -> build/lib.linux-x86_64-3.7/bsddb3
  copying Lib3/bsddb/db.py -> build/lib.linux-x86_64-3.7/bsddb3
  creating build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_distributed_transactions.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_compare.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_recno.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_dbshelve.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_db.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_dbobj.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_get_none.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_associate.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_dbtables.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_fileid.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_replication.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_sequence.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_compat.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_all.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_queue.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_join.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_pickle.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/__init__.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_lock.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_thread.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_cursor_pget_bug.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_basics.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_dbenv.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_misc.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  copying Lib3/bsddb/test/test_early_close.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
  running build_ext
  building 'bsddb3._pybsddb' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/Modules
  /home/mike/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mike/miniconda3/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mike/miniconda3/include -fPIC -I/usr/include -I/home/mike/miniconda3/include/python3.7m -c Modules/_bsddb.c -o build/temp.linux-x86_64-3.7/Modules/_bsddb.o
  In file included from /home/mike/miniconda3/include/python3.7m/Python.h:11:0,
                   from Modules/_bsddb.c:94:
  /usr/include/limits.h:26:10: fatal error: bits/libc-header-start.h: No such file or directory
   #include <bits/libc-header-start.h>
            ^~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command '/home/mike/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for bsddb3
  Running setup.py clean for bsddb3
Failed to build bsddb3
Installing collected packages: bsddb3, gutenberg
    Running setup.py install for bsddb3 ... error
    ERROR: Command errored out with exit status 1:
     command: /home/mike/miniconda3/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ppo71pv/install-record.txt --single-version-externally-managed --compile --install-headers /home/mike/miniconda3/include/python3.7m/bsddb3
         cwd: /tmp/pip-install-j0827hki/bsddb3/
    Complete output (56 lines):
    Found Berkeley DB 5.3 installation.
      include files in /usr/include
      library files in /usr/lib/x86_64-linux-gnu
      library name is libdb-5.3
    Detected Berkeley DB version 5.3 from db.h
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/dbutils.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/dbrecio.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/dbshelve.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/dbtables.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/dbobj.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/__init__.py -> build/lib.linux-x86_64-3.7/bsddb3
    copying Lib3/bsddb/db.py -> build/lib.linux-x86_64-3.7/bsddb3
    creating build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_distributed_transactions.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_compare.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_recno.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_dbshelve.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_db.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_dbobj.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_get_none.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_associate.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_dbtables.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_fileid.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_replication.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_sequence.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_compat.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_all.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_queue.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_join.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_pickle.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/__init__.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_lock.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_thread.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_cursor_pget_bug.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_basics.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_dbenv.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_misc.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    copying Lib3/bsddb/test/test_early_close.py -> build/lib.linux-x86_64-3.7/bsddb3/tests
    running build_ext
    building 'bsddb3._pybsddb' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/Modules
    /home/mike/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -Wstrict-prototypes -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/mike/miniconda3/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/mike/miniconda3/include -fPIC -I/usr/include -I/home/mike/miniconda3/include/python3.7m -c Modules/_bsddb.c -o build/temp.linux-x86_64-3.7/Modules/_bsddb.o
    In file included from /home/mike/miniconda3/include/python3.7m/Python.h:11:0,
                     from Modules/_bsddb.c:94:
    /usr/include/limits.h:26:10: fatal error: bits/libc-header-start.h: No such file or directory
     #include <bits/libc-header-start.h>
              ^~~~~~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command '/home/mike/miniconda3/bin/x86_64-conda_cos6-linux-gnu-cc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/mike/miniconda3/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j0827hki/bsddb3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_ppo71pv/install-record.txt --single-version-externally-managed --compile --install-headers /home/mike/miniconda3/include/python3.7m/bsddb3 Check the logs for full command output.
mikechen66 commented 4 years ago

I solve the installation problem with the following commands in Anaconda/Conda environment. It also adapts to other virtual environment.

Part One. Terminal Commands

1. Install berkeley-db

wget http://download.oracle.com/berkeley-db/db-4.8.30.zip unzip db-4.8.30.zip cd db-4.8.30/ cd build_unix/ ../dist/configure --prefix=/usr/local --enable-cxx make -j12 make install sudo make install

2. Install bsddb3

wget https://files.pythonhosted.org/packages/e9/fc/ebfbd4de236b493f9ece156f816c21df0ae87ccc22604c5f9b664efef1b9/bsddb3-6.2.6.tar.gz tar -xvzf bsddb3-6.2.6.tar.gz cd bsddb3-6.2.6/ python setup.py install --berkeley-db=/usr/local/

3. Installl gutenberg

pip install gutenberg

Part Two. Actual Installation Procedure

Since the commands are complex, I am pleased to show the actual installation procedure to enable the sound understanding.

1. Install berkeley-db

(base) mike@mc:~$ wget http://download.oracle.com/berkeley-db/db-4.8.30.zip (base) mike@mc:~$ unzip db-4.8.30.zip (base) mike@mc:~$ cd db-4.8.30/ (base) mike@mc:~/db-4.8.30$ cd build_unix/ (base) mike@mc:~/db-4.8.30/build_unix$ ../dist/configure --prefix=/usr/local --enable-cxx (base) mike@mc:~/db-4.8.30/build_unix$ make -j12 (base) mike@mc:~/db-4.8.30/build_unix$ make install (base) mike@mc:~/db-4.8.30/build_unix$ sudo make install

The above commands enable to install the libraries that the following bsddb3 needs.

2. Install bsddb3-6.2.6

(base) mike@mc:~/db-4.8.30/build_unix$ wget https://files.pythonhosted.org/packages/e9/fc/ebfbd4de236b493f9ece156f816c21df0ae87ccc22604c5f9b664efef1b9/bsddb3-6.2.6.tar.gz (base) mike@mc:~/db-4.8.30/build_unix$ tar -xvzf bsddb3-6.2.6.tar.gz (base) mike@mc:~/db-4.8.30/build_unix$ cd bsddb3-6.2.6/ (base) mike@mc:~/db-4.8.30/build_unix/bsddb3-6.2.6$ python setup.py install --berkeley-db=/usr/local/

After completing the installation, it will show the following message.

Processing dependencies for bsddb3==6.2.6 Finished processing dependencies for bsddb3==6.2.6

3. Install gutenberg

(base) mike@mc:~/db-4.8.30/build_unix/bsddb3-6.2.6$ cd (base) mike@mc:~$ pip install gutenberg

After completing the installation, it will show the following message.

Successfully built gutenberg Installing collected packages: gutenberg Successfully installed gutenberg-0.8.1

Notes:

If the command "make install" fails, the command "sudo make install" enables a success installation.

Best regards,

Mike

c-w commented 4 years ago

Thanks for reporting your fix. Closing this issue as it seems that your problem is now resolved. Feel free to re-open if this is not the case.