search

Scifabric / pybossa

PYBOSSA is the ultimate crowdsourcing framework (aka microtasking) to analyze or enrich data that can't be processed by machines alone.
http://pybossa.com
GNU Affero General Public License v3.0
745 stars 268 forks source link

Fail to install with Vagrant #1974

Closed lukostaz closed 4 years ago

lukostaz commented 4 years ago

Description While installing pybossa with Vagrant, as described in the docs, I encounter the following problem:

    default: TASK [create PYBOSSA DB] *******************************************************
    default: changed: [127.0.0.1]
    default: 
    default: TASK [populate PYBOSSA DB] *****************************************************
    default: fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/home/vagrant/pybossa-env/bin/python /vagrant/cli.py db_create", "msg": "[Errno 2] No such file or directory", "rc": 2}
    default:    to retry, use: --limit @/vagrant/provisioning/playbook.retry
    default: 
    default: PLAY RECAP *********************************************************************
    default: 127.0.0.1                  : ok=22   changed=14   unreachable=0    failed=1   
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

Steps to Reproduce

git clone --recursive https://github.com/Scifabric/pybossa.git
cd pybossa
vagrant up

Environment Details

Not sure if related to #1965.
Could you please take a look, thank you!

teleyinex commented 4 years ago

Hi,

It looks like your Vagrant is not seeing the files:

fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/home/vagrant/pybossa-env/bin/python /vagrant/cli.py db_create", "msg": "[Errno 2] No such file or directory", "rc": 2}

Can you check that you have run vagrant up from inside the pybossa folder?

reynoldsnlp commented 4 years ago

@teleyinex I have the same problem, and I can confirm that I did a new clone of pybossa, and then vagrant up.

VirtualBox version 6.0.0 vagrant version 2.2.7

reynoldsnlp commented 4 years ago

I tried to poke around in vagrant ssh and it looks like the pybossa-env virtual environment is not there.

vagrant@vagrant:~$ ls -a
.  ..  .bash_logout  .bashrc  .cache  .gnupg  .profile  .ssh  .sudo_as_admin_successful  .vbox_version  .wget-hsts
reynoldsnlp commented 4 years ago

Looks like a typo. Perhaps the cmd should not have the /home, and should be /vagrant/pybossa-env/bin/python /vagrant/cli.py db_create.

/vagrant/pybossa-env/bin/python does exist in the VM filesystem.

reynoldsnlp commented 4 years ago

I can't find any reference to the path /vagrant/pybossa-env/bin/python in the pybossa git repository, so I'm not sure where this is coming from. Someone more familiar with vagrant and ansible will have to help out.

reynoldsnlp commented 4 years ago

I tried backing up a few commits to see if it was something recent that broke it. I checked out f18052cb6d427b79c4c5b02336f3017e42e8102a and got a completely different error:

    default: TASK [upgrade pip itself] ******************************************************
    default: ok: [127.0.0.1]
    default:
    default: TASK [install PyBossa virtualenv packages, can take some time...] **************
    default: fatal: [127.0.0.1]: FAILED! => {"changed": false, "cmd": "/home/vagrant/pybossa-env/bin/pip install -r /vagrant/requirements.txt", "failed": true, "msg": "stdout: Looking in indexes: https://pypi.python.org/simple/\nObtaining file:///vagrant (from -r /vagrant/requirements.txt (line 3))\nCollecting alembic==0.9.10\n  Downloading alembic-0.9.10.tar.gz (1.0 MB)\nCollecting arrow==0.12.1\n  Downloading arrow-0.12.1.tar.gz (65 kB)\nCollecting asn1crypto==0.24.0\n  Downloading asn1crypto-0.24.0-py2.py3-none-any.whl (101 kB)\nCollecting Babel==2.6.0\n  Downloading Babel-2.6.0-py2.py3-none-any.whl (8.1 MB)\nCollecting beautifulsoup4==4.7.0\n  Downloading beautifulsoup4-4.7.0-py2-none-any.whl (94 kB)\nCollecting blinker==1.4\n  Downloading blinker-1.4.tar.gz (111 kB)\nCollecting certifi==2018.11.29\n  Downloading certifi-2018.11.29-py2.py3-none-any.whl (154 kB)\nCollecting cffi>=1.11.5\n  Downloading cffi-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (387 kB)\nCollecting chardet==3.0.4\n  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)\nCollecting Click==7.0\n  Downloading Click-7.0-py2.py3-none-any.whl (81 kB)\nCollecting colorama==0.4.1\n  Downloading colorama-0.4.1-py2.py3-none-any.whl (15 kB)\nCollecting cov-core==1.15.0\n  Downloading cov-core-1.15.0.tar.gz (5.9 kB)\nCollecting coverage==4.5.2\n  Downloading coverage-4.5.2-cp27-cp27mu-manylinux1_x86_64.whl (201 kB)\nCollecting croniter==0.3.28\n  Downloading croniter-0.3.28-py2.py3-none-any.whl (16 kB)\nCollecting cryptography>=2.4.2\n  Downloading cryptography-2.9-cp27-cp27mu-manylinux2010_x86_64.whl (2.7 MB)\nCollecting cssselect==1.0.3\n  Downloading cssselect-1.0.3-py2.py3-none-any.whl (16 kB)\nCollecting debtcollector==1.20.0\n  Downloading debtcollector-1.20.0-py2.py3-none-any.whl (18 kB)\nCollecting decorator==4.3.0\n  Downloading decorator-4.3.0-py2.py3-none-any.whl (9.2 kB)\nCollecting entrypoints==0.2.3\n  Downloading entrypoints-0.2.3-py2.py3-none-any.whl (9.4 kB)\nCollecting factory-boy==2.4.1\n  Downloading factory_boy-2.4.1.tar.gz (78 kB)\nCollecting Faker==1.0.1\n  Downloading Faker-1.0.1-py2.py3-none-any.whl (845 kB)\nCollecting feedparser==5.2.1\n  Downloading feedparser-5.2.1.tar.bz2 (192 kB)\nCollecting Flask==1.0.2\n  Downloading Flask-1.0.2-py2.py3-none-any.whl (91 kB)\nCollecting Flask-Assets==0.12\n  Downloading Flask-Assets-0.12.tar.gz (22 kB)\nCollecting Flask-Babel==0.9\n  Downloading Flask-Babel-0.9.tar.gz (38 kB)\nCollecting Flask-Cors==3.0.2\n  Downloading Flask_Cors-3.0.2-py2.py3-none-any.whl (15 kB)\nCollecting Flask-DebugToolbar==0.10.1\n  Downloading Flask_DebugToolbar-0.10.1-py2.py3-none-any.whl (326 kB)\nCollecting Flask-HTTPAuth==3.2.4\n  Downloading Flask_HTTPAuth-3.2.4-py2.py3-none-any.whl (3.8 kB)\nCollecting flask-json-multidict==1.0.0\n  Downloading flask_json_multidict-1.0.0.tar.gz (10 kB)\nCollecting Flask-Login==0.4.1\n  Downloading Flask-Login-0.4.1.tar.gz (14 kB)\nCollecting Flask-Mail==0.9.1\n  Downloading Flask-Mail-0.9.1.tar.gz (45 kB)\nCollecting Flask-Misaka==0.3.0\n  Downloading Flask-Misaka-0.3.0.tar.gz (168 kB)\nCollecting Flask-OAuthlib==0.9.1\n  Downloading Flask_OAuthlib-0.9.1-py2.py3-none-any.whl (38 kB)\nCollecting Flask-Plugins==1.6.1\n  Downloading Flask-Plugins-1.6.1.tar.gz (10 kB)\nCollecting flask-profiler==1.6\n  Downloading flask_profiler-1.6.tar.gz (327 kB)\nCollecting Flask-SimpleLDAP==1.1.2\n  Downloading Flask-SimpleLDAP-1.1.2.tar.gz (4.6 kB)\nCollecting Flask-SQLAlchemy==2.3.2\n  Downloading Flask_SQLAlchemy-2.3.2-py2.py3-none-any.whl (16 kB)\nCollecting Flask-WTF>=0.9.5\n  Downloading Flask_WTF-0.14.3-py2.py3-none-any.whl (13 kB)\nCollecting flatten-json==0.1.6\n  Downloading flatten_json-0.1.6.tar.gz (3.3 kB)\nCollecting google-api-python-client==1.5.5\n  Downloading google_api_python_client-1.5.5-py2.py3-none-any.whl (51 kB)\nCollecting html2text==2014.7.3\n  Downloading html2text-2014.7.3.tar.gz (32 kB)\nCollecting httplib2==0.12.0\n  Downloading httplib2-0.12.0.tar.gz (218 kB)\nCollecting humanize==0.5.1\n  Downloading humanize-0.5.1.tar.gz (14 kB)\nCollecting idna==2.8\n  Downloading idna-2.8-py2.py3-none-any.whl (58 kB)\nCollecting iiif-prezi>=0.2.9\n  Downloading iiif-prezi-0.3.0.tar.gz (27 kB)\nCollecting infinity==1.4\n  Downloading infinity-1.4.tar.gz (5.4 kB)\nCollecting intervals==0.8.1\n  Downloading intervals-0.8.1.tar.gz (15 kB)\nCollecting iso8601==0.1.12\n  Downloading iso8601-0.1.12-py2.py3-none-any.whl (12 kB)\nCollecting itsdangerous==1.1.0\n  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)\n\n:stderr: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support\nERROR: Could not find a version that satisfies the requirement jeepney==0.4 (from pybossa==3.1.2->-r /vagrant/requirements.txt (line 3)) (from versions: none)\nERROR: No matching distribution found for jeepney==0.4 (from pybossa==3.1.2->-r /vagrant/requirements.txt (line 3))\n"}
    default:
    default: PLAY RECAP *********************************************************************
    default: 127.0.0.1                  : ok=15   changed=8    unreachable=0    failed=1
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.
teleyinex commented 4 years ago

Hi,

I can confirm that's failing when you run it for the first time, however I've managed to make it work by after just failing, run the following command:

vagrant provision

That will copy the files, and you will be able to run it. The previous command would make it work, and then you will need the following:

vagrant ssh
source pybossa-env/bin/activate
python run.py

That should start the server. Can you try again? Please, use the latest master branch.

reynoldsnlp commented 4 years ago

I will try that now.

In the meantime, I found the following repository that uses docker-compose to spin up an instance, and it worked perfectly: https://github.com/jvstein/docker-compose-pybossa. It would probably be nice to have something like this in your official repository, too. It seems like vagrant is losing territory to docker in the devops world anyway.

reynoldsnlp commented 4 years ago

@teleyinex That almost worked. I'm getting an SQL error now:

 * Serving Flask app "pybossa.core" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
[2020-04-13 20:17:54,625] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    cursor, statement, parameters, context
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedTable: relation "project" does not exist
LINE 4:            FROM project, "user"
                        ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/vagrant/pybossa/view/home.py", line 41, in home
    tmp_projects = cached_projects.get_featured('featured', page, per_page)
  File "/vagrant/pybossa/cache/projects.py", line 280, in get_featured
    return get_all_featured()[offset:offset+per_page]
  File "/vagrant/pybossa/cache/__init__.py", line 117, in wrapper
    output = f(*args, **kwargs)
  File "/vagrant/pybossa/cache/projects.py", line 260, in get_all_featured
    results = session.execute(sql)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 162, in do
    return getattr(self.registry(), name)(*args, **kwargs)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1278, in execute
    clause, params or {}
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 984, in execute
    return meth(self, multiparams, params)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement
    distilled_params,
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context
    e, statement, parameters, cursor, context
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    cursor, statement, parameters, context
  File "/vagrant/pybossa-env/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "project" does not exist
LINE 4:            FROM project, "user"
                        ^

[SQL: SELECT project.id, project.name, project.short_name, project.info,
               project.created, project.updated, project.description,
               "user".fullname AS owner
           FROM project, "user"
           WHERE project.featured=true
           AND "user".id=project.owner_id
           AND "user".restrict=false
           GROUP BY project.id, "user".id;]
(Background on this error at: http://sqlalche.me/e/f405)
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET / HTTP/1.1" 500 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/css/gen/pybossa.min.css?f7bfc731 HTTP/1.1" 200 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/img/default_logo.svg HTTP/1.1" 200 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/img/pybossa.svg HTTP/1.1" 200 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/js/gen/default.min.js?83439d70 HTTP/1.1" 200 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/img/favicon/manifest.json HTTP/1.1" 200 -
10.0.2.2 - - [13/Apr/2020 20:17:55] "GET /static/img/favicon/favicon.ico HTTP/1.1" 200 -
teleyinex commented 4 years ago

Hi,

Regarding docker: yes, it's in the todo list, but for now we don't have enough time to build it. You can find some docker solutions based on PYBOSSA in docker hub (some of them probably pretty old). In any case, feel free to send a Dockerfile to get it working and we will merge it without problems.

Regarding your vagrant installation: it looks like the issue is related to the DB not having created the required tables. In that case, ssh into the VM and from there run the command:

python cli.py db_create

That should do the trick!

teleyinex commented 4 years ago

Closing it as there is no more feedback. Please re-open if you need it.