search

frappe / bench

CLI to manage Multi-tenant deployments for Frappe apps
https://frappeframework.com/docs/user/en/bench
GNU General Public License v3.0
1.4k stars 1.21k forks source link

Issue easy installing bench on ubuntu 16.04 (AWS EC2 and DigitalOcean) #322

Closed chdecultot closed 8 years ago

chdecultot commented 8 years ago

Hi all,

I have been trying to install bench today on two different machines: an AWS EC2 (Ubuntu 16.04 64bit/ 1 GB RAM) and a DigitalOcean Droplet (Ubuntu 16.04 64bit/ 2GB RAM).

Both machines were clean servers on which I run the exact following process with a non-root user ("Easy Install" process):

sudo apt-get update && sudo apt-get upgrade sudo apt-get install python-minimal wget https://raw.githubusercontent.com/frappe/bench/master/playbooks/install.py sudo -H python install.py --develop

But the install returns the same error as #285 which is already closed:

TASK [init bench] **************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["bench", "init", "/home/ch/frappe-bench", "--frappe-branch", "develop"], "delta": "0:01:33.063686", "end": "2016-10-05 16:08:14.131978", "failed": true, "rc": 1, "start": "2016-10-05 16:06:41.068292", "stderr": "INFO:bench.app:getting app frappe\nCloning into 'frappe'...\nINFO:bench.app:installing frappe\nCommand \"/home/ch/frappe-bench/env/bin/python -u -c \"import setuptools, tokenize;__file__='/tmp/pip-build-F0TDiD/python-ldap/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))\" install --record /tmp/pip-cT10Xq-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ch/frappe-bench/env/include/site/python2.7/python-ldap\" failed with error code 1 in /tmp/pip-build-F0TDiD/python-ldap/\nTraceback (most recent call last):\n  File \"/usr/local/bin/bench\", line 11, in <module>\n    load_entry_point('bench', 'console_scripts', 'bench')()\n  File \"/home/ch/.bench/bench/cli.py\", line 40, in cli\n    bench_command()\n  File \"/usr/local/lib/python2.7/dist-packages/click/core.py\", line 716, in __call__\n    return self.main(*args, **kwargs)\n  File \"/usr/local/lib/python2.7/dist-packages/click/core.py\", line 696, in main\n    rv = self.invoke(ctx)\n  File \"/usr/local/lib/python2.7/dist-packages/click/core.py\", line 1060, in invoke\n    return _process_result(sub_ctx.command.invoke(sub_ctx))\n  File \"/usr/local/lib/python2.7/dist-packages/click/core.py\", line 889, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File \"/usr/local/lib/python2.7/dist-packages/click/core.py\", line 534, in invoke\n    return callback(*args, **kwargs)\n  File \"/home/ch/.bench/bench/commands/make.py\", line 19, in init\n    verbose=verbose, clone_from=clone_from)\n  File \"/home/ch/.bench/bench/utils.py\", line 56, in init\n    get_app(frappe_path, branch=frappe_branch, bench_path=path, build_asset_files=False, verbose=verbose)\n  File \"/home/ch/.bench/bench/app.py\", line 73, in get_app\n    install_app(app=app_name, bench_path=bench_path, verbose=verbose)\n  File \"/home/ch/.bench/bench/app.py\", line 103, in install_app\n    find_links=find_links))\n  File \"/home/ch/.bench/bench/utils.py\", line 127, in exec_cmd\n    raise CommandFailedError(cmd)\nbench.utils.CommandFailedError: /home/ch/frappe-bench/env/bin/pip install -q  -e /home/ch/frappe-bench/apps/frappe", "stdout": "Already using interpreter /usr/bin/python\ninstalling frappe", "stdout_lines": ["Already using interpreter /usr/bin/python", "installing frappe"], "warnings": []}
    to retry, use: --limit @develop/install.retry

PLAY RECAP *********************************************************************
localhost                  : ok=26   changed=15   unreachable=0    failed=1   

Traceback (most recent call last):
  File "install.py", line 349, in <module>
    install_bench(args)
  File "install.py", line 108, in install_bench
    run_playbook('develop/install.yml', sudo=True, extra_vars=extra_vars)
  File "install.py", line 299, in run_playbook
    success = subprocess.check_call(args, cwd=os.path.join(cwd, 'playbooks'))
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['ansible-playbook', '-c', 'local', 'develop/install.yml', '-e', '@/tmp/extra_vars.json', '--become', '--become-user=ch']' returned non-zero exit status 2

When trying to install again (sudo -H python install.py --develop), the installation seem to work:

PLAY RECAP *********************************************************************
localhost                  : ok=33   changed=6    unreachable=0    failed=0   

    Frappe/ERPNext has been successfully installed on your machine.

    Please find mysql root password and admin password in "passwords.txt" in your root directory,
    you can remove the file after making note of the passwords.

And then when trying to start the bench bench start in the frappe-bench folder, I get the following error:

16:09:58 system           | redis_socketio.1 started (pid=3823)
16:09:58 system           | worker_long.1 started (pid=3824)
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.362 * Increased maximum number of open files to 10032 (it was originally set to 1024).
16:09:58 redis_socketio.1 |                 _._                                                  
16:09:58 redis_socketio.1 |            _.-``__ ''-._                                             
16:09:58 redis_socketio.1 |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
16:09:58 redis_socketio.1 |   .-`` .-```.  ```\/    _.,_ ''-._                                   
16:09:58 redis_socketio.1 |  (    '      ,       .-`  | `,    )     Running in standalone mode
16:09:58 redis_socketio.1 |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 12000
16:09:58 redis_socketio.1 |  |    `-._   `._    /     _.-'    |     PID: 3825
16:09:58 redis_socketio.1 |   `-._    `-._  `-./  _.-'    _.-'                                   
16:09:58 redis_socketio.1 |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_socketio.1 |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
16:09:58 redis_socketio.1 |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_socketio.1 |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_socketio.1 |  |    `-._`-._        _.-'_.-'    |                                  
16:09:58 redis_socketio.1 |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_socketio.1 |       `-._    `-.__.-'    _.-'                                       
16:09:58 redis_socketio.1 |           `-._        _.-'                                           
16:09:58 redis_socketio.1 |               `-.__.-'                                               
16:09:58 redis_socketio.1 | 
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.364 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.364 # Server started, Redis version 3.0.6
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.365 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.365 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
16:09:58 redis_socketio.1 | 3825:M 05 Oct 16:09:58.365 * The server is now ready to accept connections on port 12000
16:09:58 system           | web.1 started (pid=3835)
16:09:58 system           | watch.1 started (pid=3831)
16:09:58 system           | redis_queue.1 started (pid=3834)
16:09:58 system           | socketio.1 started (pid=3832)
16:09:58 system           | worker_default.1 started (pid=3836)
16:09:58 system           | worker_short.1 started (pid=3839)
16:09:58 system           | redis_cache.1 started (pid=3841)
16:09:58 system           | schedule.1 started (pid=3833)
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.441 * Increased maximum number of open files to 10032 (it was originally set to 1024).
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.450 * Increased maximum number of open files to 10032 (it was originally set to 1024).
16:09:58 redis_queue.1    |                 _._                                                  
16:09:58 redis_queue.1    |            _.-``__ ''-._                                             
16:09:58 redis_queue.1    |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
16:09:58 redis_queue.1    |   .-`` .-```.  ```\/    _.,_ ''-._                                   
16:09:58 redis_queue.1    |  (    '      ,       .-`  | `,    )     Running in standalone mode
16:09:58 redis_queue.1    |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 11000
16:09:58 redis_queue.1    |  |    `-._   `._    /     _.-'    |     PID: 3844
16:09:58 redis_queue.1    |   `-._    `-._  `-./  _.-'    _.-'                                   
16:09:58 redis_queue.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_queue.1    |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
16:09:58 redis_queue.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_queue.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_queue.1    |  |    `-._`-._        _.-'_.-'    |                                  
16:09:58 redis_queue.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_queue.1    |       `-._    `-.__.-'    _.-'                                       
16:09:58 redis_queue.1    |           `-._        _.-'                                           
16:09:58 redis_queue.1    |               `-.__.-'                                               
16:09:58 redis_queue.1    | 
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.468 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.468 # Server started, Redis version 3.0.6
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.468 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.468 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
16:09:58 redis_queue.1    | 3844:M 05 Oct 16:09:58.468 * The server is now ready to accept connections on port 11000
16:09:58 redis_cache.1    |                 _._                                                  
16:09:58 redis_cache.1    |            _.-``__ ''-._                                             
16:09:58 redis_cache.1    |       _.-``    `.  `_.  ''-._           Redis 3.0.6 (00000000/0) 64 bit
16:09:58 redis_cache.1    |   .-`` .-```.  ```\/    _.,_ ''-._                                   
16:09:58 redis_cache.1    |  (    '      ,       .-`  | `,    )     Running in standalone mode
16:09:58 redis_cache.1    |  |`-._`-...-` __...-.``-._|'` _.-'|     Port: 13000
16:09:58 redis_cache.1    |  |    `-._   `._    /     _.-'    |     PID: 3854
16:09:58 redis_cache.1    |   `-._    `-._  `-./  _.-'    _.-'                                   
16:09:58 redis_cache.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_cache.1    |  |    `-._`-._        _.-'_.-'    |           http://redis.io        
16:09:58 redis_cache.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_cache.1    |  |`-._`-._    `-.__.-'    _.-'_.-'|                                  
16:09:58 redis_cache.1    |  |    `-._`-._        _.-'_.-'    |                                  
16:09:58 redis_cache.1    |   `-._    `-._`-.__.-'_.-'    _.-'                                   
16:09:58 redis_cache.1    |       `-._    `-.__.-'    _.-'                                       
16:09:58 redis_cache.1    |           `-._        _.-'                                           
16:09:58 redis_cache.1    |               `-.__.-'                                               
16:09:58 redis_cache.1    | 
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.475 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.475 # Server started, Redis version 3.0.6
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.475 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.475 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
16:09:58 redis_cache.1    | 3854:M 05 Oct 16:09:58.475 * The server is now ready to accept connections on port 13000
16:09:59 worker_long.1    | Usage: bench [OPTIONS] COMMAND [ARGS]...
16:09:59 worker_long.1    | 
16:09:59 worker_long.1    | Error: No such command "worker".
16:09:59 system           | worker_long.1 stopped (rc=2)
16:09:59 system           | sending SIGTERM to redis_socketio.1 (pid 3823)
16:09:59 system           | sending SIGTERM to watch.1 (pid 3831)
16:09:59 system           | sending SIGTERM to redis_queue.1 (pid 3834)
16:09:59 system           | sending SIGTERM to web.1 (pid 3835)
16:09:59 system           | sending SIGTERM to schedule.1 (pid 3833)
16:09:59 system           | sending SIGTERM to worker_default.1 (pid 3836)
16:09:59 system           | sending SIGTERM to socketio.1 (pid 3832)
16:09:59 system           | sending SIGTERM to redis_cache.1 (pid 3841)
16:09:59 system           | sending SIGTERM to worker_short.1 (pid 3839)
16:09:59 redis_queue.1    | 3844:signal-handler (1475683799) Received SIGTERM scheduling shutdown...
16:09:59 redis_queue.1    | 3844:M 05 Oct 16:09:59.365 # User requested shutdown...
16:09:59 redis_queue.1    | 3844:M 05 Oct 16:09:59.365 # Redis is now ready to exit, bye bye...
16:09:59 redis_socketio.1 | 3825:signal-handler (1475683799) Received SIGTERM scheduling shutdown...
16:09:59 redis_socketio.1 | 3825:M 05 Oct 16:09:59.364 # User requested shutdown...
16:09:59 redis_socketio.1 | 3825:M 05 Oct 16:09:59.364 # Redis is now ready to exit, bye bye...
16:09:59 redis_cache.1    | 3854:signal-handler (1475683799) Received SIGTERM scheduling shutdown...
16:09:59 redis_cache.1    | 3854:M 05 Oct 16:09:59.365 # User requested shutdown...
16:09:59 redis_cache.1    | 3854:M 05 Oct 16:09:59.365 # Redis is now ready to exit, bye bye...
16:09:59 system           | redis_cache.1 stopped (rc=-15)
16:09:59 system           | redis_socketio.1 stopped (rc=-15)
16:09:59 system           | redis_queue.1 stopped (rc=-15)
16:09:59 system           | worker_default.1 stopped (rc=-15)
16:09:59 system           | worker_short.1 stopped (rc=-15)
16:09:59 system           | socketio.1 stopped (rc=-15)
16:09:59 system           | schedule.1 stopped (rc=-15)
16:09:59 system           | web.1 stopped (rc=-15)
16:09:59 system           | watch.1 stopped (rc=-15)

Several issues have been created and closed on similar issues, but it doesn't seem to work out-of-the-box as it should.

I would be grateful for any help solving this issue.

Thanks a lot in advance.

Charles-Henri

vjFaLk commented 8 years ago

Can you give me the output of git status inside the frappe and erpnext folders inside frappe-bench/apps?

chdecultot commented 8 years ago

Hi vjFalk,

The frappe folder has been created with the easy install but not the ERPNext folder:

~/frappe-bench/apps$ ls
frappe

Here is the status of the frappe folder:

~/frappe-bench/apps/frappe$ git status
On branch develop
Your branch is up-to-date with 'upstream/develop'.
nothing to commit, working directory clean
vjFaLk commented 8 years ago

What does bench worker give you? Try running from within the frappe-bench directory:

./env/bin/pip install -e ./apps/frappe and then do bench start again

Off topic : Any reason you're doing a development setup on a DO Droplet?

shreyasp commented 8 years ago

@chdecultot After failing did you install python-ldap and its dependencies manually after which had got to see successful installation message ?

vjFaLk commented 8 years ago

@shreyasp Good catch

@chdecultot Looking back at your error log. I think running sudo -H pip install python-ldap followed by ./env/bin/pip install -e ./apps/frappe from frappe-bench should do the trick!

chdecultot commented 8 years ago

Here is the result of bench worker

~/frappe-bench$ bench worker
Usage: bench [OPTIONS] COMMAND [ARGS]...

Error: No such command "worker".

When running ./env/bin/pip install -e ./apps/frappe, I have the following result:

Command "/home/ch/frappe-bench/env/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-oYaL2f/python-ldap/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-dAuNnr-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/ch/frappe-bench/env/include/site/python2.7/python-ldap" failed with error code 1 in /tmp/pip-build-oYaL2f/python-ldap/

So I installed sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev before running ./env/bin/pip install -e ./apps/frappe again (sorry I did that before you posted your next message @vjFaLk ). And this time got no error.

So I continued the installation of ERPNext manually and it worked fine:

bench get-app erpnext https://github.com/frappe/erpnext
bench new-site site1.local
bench --site site1.local install-app erpnext
bench start

You were right @shreyasp , the problem was that python-ldap was missing.

Would it be possible to update the easy install script to avoid other people having a similar issue?

And @vjFaLk , to answer your question regarding a development environment on DO, I have just made an habit of setting up all my development workflow on remote servers. It is just more convenient for me :-)

Thanks a lot for your help guys.

Best regards, Charles-Henri

shreyasp commented 8 years ago

@chdecultot I am making PR with dependencies today itself. Thanks for reporting the issue.

chdecultot commented 8 years ago

Thanks a lot @shreyasp, I have successfully retested on an AWS-EC2 Machine with Ubuntu 16.04 (64bit):

image

Therefore this issue can be closed.