Errors when trying to install Core Lightning on Ubuntu

[michaelfolkson]

Tried to install Core Lightning (master) on a fresh Ubuntu 22.04 LTS x64

pip3 install -r plugins/clnrest/requirements.txt

hit an error:

ERROR: Cannot uninstall 'blinker'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

make successfully completes but then when I try running lightningd --network=signet with a fully synced bitcoind -signet on the same server I get:

2023-08-07T17:58:39.545Z INFO    plugin-clnrest.py: Killing plugin: disabled itself: No module named 'flask'
bitcoin-cli exec failed: Bad file descriptor2023-08-07T17:58:45.713Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
2023-08-07T17:58:45.713Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

I try to pip3 install flask but that hits the blinker errror

ERROR: Cannot uninstall 'blinker'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

I did try

 pip3 install --ignore-installed blinker

which successfully installs blinker.

But...

./lightningd --network=signet

gets an error:

2023-08-07T18:03:23.759Z INFO    plugin-clnrest.py: Killing plugin: disabled itself: No module named 'flask'
bitcoin-cli exec failed: Bad file descriptor2023-08-07T18:03:23.789Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
2023-08-07T18:03:23.789Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

pip3 install flask flask_restx flask_socketio

Then:

./lightningd --network=signet

gets an error:

2023-08-07T18:04:29.516Z INFO    plugin-clnrest.py: Killing plugin: disabled itself: No module named 'pyln.client'
bitcoin-cli exec failed: Bad file descriptor2023-08-07T18:04:29.542Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
2023-08-07T18:04:29.542Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

pip3 install pyln-client

Then

./lightningd --network=signet

gets this error:

File "/root/lightning/plugins/clnrest/clnrest.py", line 32, in <module>
    socketio = SocketIO(app, async_mode="gevent", cors_allowed_origins="*")
  File "/usr/local/lib/python3.10/dist-packages/flask_socketio/__init__.py", line 187, in __init__
    self.init_app(app, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/flask_socketio/__init__.py", line 243, in init_app
    self.server = socketio.Server(**self.server_options)
  File "/usr/local/lib/python3.10/dist-packages/socketio/server.py", line 134, in __init__
    self.eio = self._engineio_server_class()(**engineio_options)
  File "/usr/local/lib/python3.10/dist-packages/engineio/server.py", line 149, in __init__
    raise ValueError('Invalid async_mode specified')
ValueError: Invalid async_mode specified
2023-08-07T18:10:45.414Z INFO    plugin-clnrest.py: Killing plugin: exited before replying to getmanifest
2023-08-07T18:10:45.414Z **BROKEN** plugin-clnrest.py: Plugin marked as important, shutting down lightningd!
lightningd: lightningd/plugin.c:1909: plugins_init: Assertion `ret == plugins' failed.
lightningd: FATAL SIGNAL 6 (version v23.08rc1-30-g98c805e)
0x55b2d8b6b6ca send_backtrace
    common/daemon.c:33
0x55b2d8b6b761 crashdump
    common/daemon.c:75
0x7f9ea772d51f ???
    ???:0
0x7f9ea7781a7c ???
    ???:0
0x7f9ea772d475 ???
    ???:0
0x7f9ea77137f2 ???
    ???:0
0x7f9ea771371a ???
    ???:0
0x7f9ea7724e95 ???
    ???:0
0x55b2d8b407f9 plugins_init
    lightningd/plugin.c:1909
0x55b2d8b1bbc2 main
    lightningd/lightningd.c:1079
0x7f9ea7714d8f ???
    ???:0
0x7f9ea7714e3f ???
    ???:0
0x55b2d8af7c64 ???
    ???:0
0xffffffffffffffff ???
    ???:0
Aborted (core dumped)

[ShahanaFarooqui]

The complete clnrest dependency install command is pip install json5 flask flask_restx gunicorn pyln-client flask-socketio gevent gevent-websocket. I can see flask flask_restx pyln-client flask-socketio installed but you can try running it after installing remaining dependencies ie. json5 gunicorn gevent gevent-websocket.

[michaelfolkson]

@ShahanaFarooqui: Thanks for this Shahana. Will see if this works. Was following these instructions where it wasn't clear that pip3 install -r plugins/clnrest/requirements.txt was optional nor that it needed the dependencies you listed. Will open a PR making those docs a bit clearer if this fixes the issue.

[ShahanaFarooqui]

@michaelfolkson Instructions for pip install -r plugins/clnrest/requirements.txt are good to follow with future releases as these dependencies are already included in plugins/clnrest/requirements.txt. But RC1 release's plugins/clnrest/requirements.txt expects pyln-proto version 23.8 and fails.

For now, I have downgraded it to 23.5 ( with RC2) and will upgrade it to 23.8 after the final release, when pyln-proto is also available on pypi. So after RC2, explicit installation should not be required anymore.

[michaelfolkson]

Installed Shahana's dependencies and built again. Now just get the bitcoin-cli error.

bitcoin-cli exec failed: Bad file descriptor2023-08-08T10:12:41.183Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
2023-08-08T10:12:41.184Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

[rustyrussell]

I did a clean install on Ubuntu 22.04 LTS x64 and could not reproduce this; I did fix the bcli report, so we might have a clue why this fails to work.

I had to do some weird Python stuff to get the packages I needed to install, but that's life on Ubuntu AFAICT.

[michaelfolkson]

Thanks for looking into this Rusty. Still getting this on latest master though:

plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
2023-08-14T08:18:37.860Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

[vincenzopalazzo]

This is strange @michaelfolkson

Could you share your core lightning config file?

[michaelfolkson]

@vincenzopalazzo: Haven't set one up so nothing in there and doesn't exist in ~/.lightning/signet

[vincenzopalazzo]

Ok ok, maybe core lightning is looking in the wrong directory for bitcoin-cli? not sure about this

Or this can be a permission problem

[michaelfolkson]

I am running as root on Digital Ocean. I'll try running as a different user and see if that makes any difference. I have Core Lightning running on a laptop fine so pretty sure I'm not doing anything dumb here....

[vincenzopalazzo]

I am running as root on Digital Ocean. I'll try running as a different user and see if that makes any difference. I have Core Lightning running on a laptop fine so pretty sure I'm not doing anything dumb here....

Yeah I think that the problem can be the permission or that core lightning is looking in another directory for bitcoin-cli

[michaelfolkson]

The problem isn't resolved by not running asroot. I get the same error running both bitcoind and lightningd as an alternative user. I'll play around with configs, paths etc and see if that can resolve it.

[vincenzopalazzo]

Another test that you can do is running bitcoin-cli yourself and see what is the result

[urza]

@michaelfolkson Did you managed to resolve the bitcoin-cli Bad file descriptor issue? I run into the same problem..

[urza]

Nevermind, my problem was I had ".config" file instead of just "config"