Experience installing from scratch (from PyPI)

[jean]

At https://opensource.zerodb.com/ there's a very tempting $ pip install zerodb-server. I decided to try it starting from vagrant's ubuntu/trusty64 box. In the following I kept just the info I looked at to find missing packages.

vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
The program 'pip' is currently not installed. To run 'pip' please ask your administrator to install the package 'python-pip'
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install python-pip
[...]
After this operation, 48.0 MB of additional disk space will be used.
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
[...]
persistent/_compat.h:18:20: fatal error: Python.h: No such file or directory
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install python-dev
[...]
After this operation, 35.1 MB of additional disk space will be used.
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
[...]
    c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install libffi-dev
[...]
After this operation, 323 kB of additional disk space will be used.
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
[...]
Compiling /tmp/pip_build_vagrant/pexpect/pexpect/async.py ...
  File "/tmp/pip_build_vagrant/pexpect/pexpect/async.py", line 16
    transport, pw = yield from asyncio.get_event_loop()\
                             ^
SyntaxError: invalid syntax
[...]

Are we pulling in a Python 3 package?

[...]
scrypt-1.1.6/lib/crypto/crypto_aesctr.c:38:25: fatal error: openssl/aes.h: No such file or directory
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install libssl-dev
[...]
After this operation, 8279 kB of additional disk space will be used.
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
[...]
Requirement already satisfied (use --upgrade to upgrade): zerodb-server in ./.local/lib/python2.7/site-packages

... but that is a lie, since the last one errored out. Let's see how far we get anyway:

vagrant@vagrant-ubuntu-trusty-64:~$ zerodb-manage initdb
[...]
ImportError: No module named ipython_genutils
[...]
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user jupyter
[...]
    build/temp.linux-x86_64-2.7/scratch/vers.c:4:17: fatal error: zmq.h: No such file or directory

OK, let's consolidate and start over:

$ vagrant snapshot go start
$ vagrant ssh
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install python-pip python-dev libssl-dev libffi-dev libzmq-dev 
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user jupyter
vagrant@vagrant-ubuntu-trusty-64:~$ pip install --user zerodb-server
vagrant@vagrant-ubuntu-trusty-64:~$ export PATH=~/.local/bin:$PATH
vagrant@vagrant-ubuntu-trusty-64:~$ zerodb-manage init_dbTraceback (most recent call last):
  File "/home/vagrant/.local/bin/zerodb-manage", line 7, in <module>
    from zerodbext.server.manage import cli
  File "/home/vagrant/.local/lib/python2.7/site-packages/zerodbext/server/manage.py", line 17, in <module>
    from zerodb import DB
  File "/home/vagrant/.local/lib/python2.7/site-packages/zerodb/__init__.py", line 1, in <module>
    from .db import DB
  File "/home/vagrant/.local/lib/python2.7/site-packages/zerodb/db.py", line 14, in <module>
    from zerodb.collective.indexing import subscribers
  File "/home/vagrant/.local/lib/python2.7/site-packages/zerodb/collective/indexing/subscribers.py", line 1, in <module>
    from zerodb.collective.indexing.queue import getQueue
  File "/home/vagrant/.local/lib/python2.7/site-packages/zerodb/collective/indexing/queue.py", line 4, in <module>
    from zope.component import getSiteManager
  File "/home/vagrant/.local/lib/python2.7/site-packages/zope/component/__init__.py", line 19, in <module>
    from zope.interface import named
ImportError: cannot import name named

Maybe the --user install is the problem?

$ vagrant snapshot go start
$ vagrant ssh
vagrant@vagrant-ubuntu-trusty-64:~$ sudo apt-get install python-pip python-dev libssl-dev libffi-dev libzmq-dev 
vagrant@vagrant-ubuntu-trusty-64:~$ sudo pip install jupyter
vagrant@vagrant-ubuntu-trusty-64:~$ sudo pip install zerodb-server
vagrant@vagrant-ubuntu-trusty-64:~$ zerodb-manage init_db
Traceback (most recent call last):
  File "/usr/local/bin/zerodb-manage", line 7, in <module>
    from zerodbext.server.manage import cli
  File "/usr/local/lib/python2.7/dist-packages/zerodbext/server/manage.py", line 17, in <module>
    from zerodb import DB
  File "/usr/local/lib/python2.7/dist-packages/zerodb/__init__.py", line 1, in <module>
    from .db import DB
  File "/usr/local/lib/python2.7/dist-packages/zerodb/db.py", line 14, in <module>
    from zerodb.collective.indexing import subscribers
  File "/usr/local/lib/python2.7/dist-packages/zerodb/collective/indexing/subscribers.py", line 1, in <module>
    from zerodb.collective.indexing.queue import getQueue
  File "/usr/local/lib/python2.7/dist-packages/zerodb/collective/indexing/queue.py", line 4, in <module>
    from zope.component import getSiteManager
  File "/usr/local/lib/python2.7/dist-packages/zope/component/__init__.py", line 19, in <module>
    from zope.interface import named
ImportError: cannot import name named

zope.interface is present at /usr/lib/python2.7/dist-packages/zope/interface.

[michwill]

Tried in docker, but not vagrant. Ubuntu in Docker has very little stuff installed by default. I think, I needed to install only pip and libffi-dev. But worths double-checking.

[michwill]

From what is in your logs, it seems like doing this exercise would help to determine what precise minimal versions of dependencies are needed. Seems like minimal version for zope.interface could be wrong

[jean]

@michwill indeed, adding pip install --user --upgrade zope.interface before installing zerodb-server makes init_db succeed. This installed version 4.2.0.

[michwill]

@jean As I understand, zope.interface is a dependency of zope.component, and zope.component is unhappy about the version installed with it. So, presumably it's a problem of zope.component, though putting zope.interface with proper version inside setup.py of zerodb wouldn't hurt

[michwill]

We are releasing new version based on ZEO5 where most of this should be re-done (will be checking when building a docker image)