YunoHost-Apps / peertube_ynh

Federated (ActivityPub) video streaming platform using P2P for YunoHost
https://joinpeertube.org/fr/
GNU Affero General Public License v3.0
88 stars 28 forks source link

Getting error 500 for peertube after rebooting the server #357

Closed yodahome closed 1 year ago

yodahome commented 1 year ago

Describe the bug

After I rebooted my YunoHost tonight (after 170 days uptime) I can no longer connect to PeerTube via the web. I get an http 500 error when I try to open the domain on any browser. First I thought the service had not come back up after the reboot, but it is running (and so is every other service on that server). In the nginx error log for the domain I get a bunch of those:

2022/11/26 22:32:09 [error] 1482#1482: 90 connect() failed (111: Connection refused) while connecting to upstream, client: 173.255.223.214, server: peertube.mydomain.de, request: "POST /inbox HTTP/1.1", upstream: "http://127.0.0.1:9000/inbox", host: "peertube.mydomain.de" 2022/11/26 22:32:11 [error] 1482#1482: 4 connect() failed (111: Connection refused) while connecting to upstream, client: 88.150.50.249, server: peertube.mydomain.de, request: "GET /socket.io/?accessToken=1610ec86fe83032e408e2289331ea4265ae8baba&EIO=4&transport=polling&t=OIrsjzn HTTP/2.0", upstream: "http://127.0.0.1:9000/socket.io/?accessToken=1610ec86fe83032e408e2289331ea4265ae8baba&EIO=4&transport=polling&t=OIrsjzn", host: "peertube.mydomain.de", referrer: "https://peertube.mydomain.de/home" 2022/11/26 22:32:11 [error] 1482#1482: 96 connect() failed (111: Connection refused) while connecting to upstream, client: 2a01:4f8:1c1e:7661::1, server: peertube.mydomain.de, request: "POST /inbox HTTP/1.1", upstream: "http://127.0.0.1:9000/inbox", host: "peertube.mydomain.de" 2022/11/26 22:57:36 [error] 1482#1482: 1747 connect() failed (111: Connection refused) while connecting to upstream, client: 88.150.50.249, server: peertube.mydomain.de, request: "GET / HTTP/2.0", upstream: "http://127.0.0.1:9000/", host: "peertube.mydomain.de" 2022/11/26 22:57:36 [error] 1482#1482: *1747 connect() failed (111: Connection refused) while connecting to upstream, client: 88.150.50.249, server: peertube.mydomain.de, request: "GET /favicon.ico HTTP/2.0", upstream: "http://127.0.0.1:9000/favicon.ico", host: "peertube.mydomain.de", referrer: "https://peertube.mydomain.de/"

I'm not sure why it would not connect to the local peertube webserver that is running on port 9000 (at least I can see it listening).

Context

Steps to reproduce

Open up any web browser, open https://peertube.mydomain.de

Expected behavior

I would expect to see the PeerTube web interface.

Logs

There are no errors that I can see in the peertube.log

Update: When looking at the startup of the service, I see this

[peertube.mydomain.de:443] 2022-11-27 01:31:45.304 error: Cannot register plugins and themes. Nov 27 01:31:45 peertube[1909]: "err": { Nov 27 01:31:45 peertube[1909]: "err": { Nov 27 01:31:45 peertube[1909]: "stack": "Error: Command failed: yarn install --pure-lockfile\nwarning package.json: No license field\nwarning No license field\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-airplay@1.1.0\" has unmet peer dependency \"video.js@>= 6.0.0\".\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-chromecast@1.3.1\" has unmet peer dependency \"video.js@>= 6 < 8\".\nwarning \"peertube-plugin-tv-streaming > babel-loader@8.2.3\" has unmet peer dependency \"webpack@>=2\".\nerror /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi: Command failed.\nExit code: 1\nCommand: node-gyp-build\nArguments: \nDirectory: /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\nOutput:\ngyp info it worked if it ends with ok\ngyp info using node-gyp@9.1.0\ngyp info using node@16.18.1 | linux | x64\ngyp info find Python using Python version 3.9.2 found at \"/usr/bin/python3\"\ngyp info spawn /usr/bin/python3\ngyp info spawn args [\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',\ngyp info spawn args 'binding.gyp',\ngyp info spawn args '-f',\ngyp info spawn args 'make',\ngyp info spawn args '-I',\ngyp info spawn args '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build/config.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/var/www/peertube/.cache/node-gyp/16.18.1/include/node/common.gypi',\ngyp info spawn args '-Dlibrary=shared_library',\ngyp info spawn args '-Dvisibility=default',\ngyp info spawn args '-Dnode_root_dir=/var/www/peertube/.cache/node-gyp/16.18.1',\ngyp info spawn args '-Dnode_gyp_dir=/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp',\ngyp info spawn args '-Dnode_lib_file=/var/www/peertube/.cache/node-gyp/16.18.1/<(target_arch)/node.lib',\ngyp info spawn args '-Dmodule_root_dir=/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi',\ngyp info spawn args '-Dnode_engine=v8',\ngyp info spawn args '--depth=.',\ngyp info spawn args '--no-parallel',\ngyp info spawn args '--generator-output',\ngyp info spawn args 'build',\ngyp info spawn args '-Goutput_dir=.'\ngyp info spawn args ]\ngyp info spawn make\ngyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]\nmake: Entering directory '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build'\ngyp ERR! build error \ngyp ERR! stack Error: make failed with exit code: null\ngyp ERR! stack at ChildProcess.onExit (/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:201:23)\ngyp ERR! stack at ChildProcess.emit (node:events:513:28)\ngyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)\ngyp ERR! System Linux 5.10.0-19-amd64\ngyp ERR! command \"/opt/node_n/n/versions/node/16.18.1/bin/node\" \"/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"\ngyp ERR! cwd /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\ngyp ERR! node -v v16.18.1\ngyp ERR! node-gyp -v v9.1.0\ngyp ERR! not ok\n\n at ChildProcess.exithandler (node:child_process:402:12)\n at ChildProcess.emit (node:events:513:28)\n at maybeClose (node:internal/child_process:1100:16)\n at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)", Nov 27 01:31:45 peertube[1909]: "message": "Command failed: yarn install --pure-lockfile\nwarning package.json: No license field\nwarning No license field\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-airplay@1.1.0\" has unmet peer dependency \"video.js@>= 6.0.0\".\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-chromecast@1.3.1\" has unmet peer dependency \"video.js@>= 6 < 8\".\nwarning \"peertube-plugin-tv-streaming > babel-loader@8.2.3\" has unmet peer dependency \"webpack@>=2\".\nerror /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi: Command failed.\nExit code: 1\nCommand: node-gyp-build\nArguments: \nDirectory: /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\nOutput:\ngyp info it worked if it ends with ok\ngyp info using node-gyp@9.1.0\ngyp info using node@16.18.1 | linux | x64\ngyp info find Python using Python version 3.9.2 found at \"/usr/bin/python3\"\ngyp info spawn /usr/bin/python3\ngyp info spawn args [\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',\ngyp info spawn args 'binding.gyp',\ngyp info spawn args '-f',\ngyp info spawn args 'make',\ngyp info spawn args '-I',\ngyp info spawn args '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build/config.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/var/www/peertube/.cache/node-gyp/16.18.1/include/node/common.gypi',\ngyp info spawn args '-Dlibrary=shared_library',\ngyp info spawn args '-Dvisibility=default',\ngyp info spawn args '-Dnode_root_dir=/var/www/peertube/.cache/node-gyp/16.18.1',\ngyp info spawn args '-Dnode_gyp_dir=/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp',\ngyp info spawn args '-Dnode_lib_file=/var/www/peertube/.cache/node-gyp/16.18.1/<(target_arch)/node.lib',\ngyp info spawn args '-Dmodule_root_dir=/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi',\ngyp info spawn args '-Dnode_engine=v8',\ngyp info spawn args '--depth=.',\ngyp info spawn args '--no-parallel',\ngyp info spawn args '--generator-output',\ngyp info spawn args 'build',\ngyp info spawn args '-Goutput_dir=.'\ngyp info spawn args ]\ngyp info spawn make\ngyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]\nmake: Entering directory '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build'\ngyp ERR! build error \ngyp ERR! stack Error: make failed with exit code: null\ngyp ERR! stack at ChildProcess.onExit (/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:201:23)\ngyp ERR! stack at ChildProcess.emit (node:events:513:28)\ngyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)\ngyp ERR! System Linux 5.10.0-19-amd64\ngyp ERR! command \"/opt/node_n/n/versions/node/16.18.1/bin/node\" \"/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"\ngyp ERR! cwd /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\ngyp ERR! node -v v16.18.1\ngyp ERR! node-gyp -v v9.1.0\ngyp ERR! not ok\n", Nov 27 01:31:45 peertube[1909]: "code": 1, Nov 27 01:31:45 peertube[1909]: "killed": false, Nov 27 01:31:45 peertube[1909]: "signal": null, Nov 27 01:31:45 peertube[1909]: "cmd": "yarn install --pure-lockfile" Nov 27 01:31:45 peertube[1909]: }, Nov 27 01:31:45 peertube[1909]: "stdout": "yarn install v1.22.19\n[1/4] Resolving packages...\n[2/4] Fetching packages...\n[3/4] Linking dependencies...\n[4/4] Building fresh packages...\ninfo Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.\n", Nov 27 01:31:45 peertube[1909]: "stderr": "warning package.json: No license field\nwarning No license field\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-airplay@1.1.0\" has unmet peer dependency \"video.js@>= 6.0.0\".\nwarning \"peertube-plugin-tv-streaming > @silvermine/videojs-chromecast@1.3.1\" has unmet peer dependency \"video.js@>= 6 < 8\".\nwarning \"peertube-plugin-tv-streaming > babel-loader@8.2.3\" has unmet peer dependency \"webpack@>=2\".\nerror /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi: Command failed.\nExit code: 1\nCommand: node-gyp-build\nArguments: \nDirectory: /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\nOutput:\ngyp info it worked if it ends with ok\ngyp info using node-gyp@9.1.0\ngyp info using node@16.18.1 | linux | x64\ngyp info find Python using Python version 3.9.2 found at \"/usr/bin/python3\"\ngyp info spawn /usr/bin/python3\ngyp info spawn args [\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',\ngyp info spawn args 'binding.gyp',\ngyp info spawn args '-f',\ngyp info spawn args 'make',\ngyp info spawn args '-I',\ngyp info spawn args '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build/config.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',\ngyp info spawn args '-I',\ngyp info spawn args '/var/www/peertube/.cache/node-gyp/16.18.1/include/node/common.gypi',\ngyp info spawn args '-Dlibrary=shared_library',\ngyp info spawn args '-Dvisibility=default',\ngyp info spawn args '-Dnode_root_dir=/var/www/peertube/.cache/node-gyp/16.18.1',\ngyp info spawn args '-Dnode_gyp_dir=/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp',\ngyp info spawn args '-Dnode_lib_file=/var/www/peertube/.cache/node-gyp/16.18.1/<(target_arch)/node.lib',\ngyp info spawn args '-Dmodule_root_dir=/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi',\ngyp info spawn args '-Dnode_engine=v8',\ngyp info spawn args '--depth=.',\ngyp info spawn args '--no-parallel',\ngyp info spawn args '--generator-output',\ngyp info spawn args 'build',\ngyp info spawn args '-Goutput_dir=.'\ngyp info spawn args ]\ngyp info spawn make\ngyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]\nmake: Entering directory '/home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi/build'\ngyp ERR! build error \ngyp ERR! stack Error: make failed with exit code: null\ngyp ERR! stack at ChildProcess.onExit (/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:201:23)\ngyp ERR! stack at ChildProcess.emit (node:events:513:28)\ngyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:293:12)\ngyp ERR! System Linux 5.10.0-19-amd64\ngyp ERR! command \"/opt/node_n/n/versions/node/16.18.1/bin/node\" \"/opt/node_n/n/versions/node/16.18.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"\ngyp ERR! cwd /home/yunohost.app/peertube/storage/plugins/node_modules/ffi-napi\ngyp ERR! node -v v16.18.1\ngyp ERR! node-gyp -v v9.1.0\ngyp ERR! not ok\n" Nov 27 01:31:45 peertube[1909]: }

It seems like something is wrong with the plugins, right? Any hints would be appreciated.

fflorent commented 1 year ago

It seems like something is wrong with the plugins, right? Any hints would be appreciated.

It seems like. It's also hard to tell which one is the culprit and why precisely (except it's somehow related to FFI).

I'd say go for first removing unwanted plugins:

$ sudo su
# su peertube -s /bin/bash
$ cat /home/yunohost.app/peertube/storage/plugins/package.json # check the "dependencies", these are the plugins you installed
$ cd /var/www/peertube
$ export PATH="/opt/node_n/n/versions/node/16.18.1/bin/:$PATH" # this put the node binaries in the $PATH variables so you call the same node version than the one used to run peertube
$ npm run plugin:uninstall -- -n <paste here the plugin name from package.json>

Then in another terminal:

$ sudo systemctl restart peertube
$ sudo journalctl -f -u peertube # to check the logs from peertube

Check the logs from journalctl and if it still fails, run another npm run plugin:uninstall -- -n ... command and restart anew your PeerTube instance until it finally start.

yodahome commented 1 year ago

Thanks, I did look into that and apparently something is wrong with the hcaptcha plugin that I had installed. Or at least, that was the last thing I toyed with. So I went to the plugins folder and tried the "yarn install --pure-lockfile"-command as the peertube user manually. That fixed it and I could access the web interface again. However after installing and uninstalling another captcha plugin (mtcaptcha) it broke again. They generally don't seem to show up during registration as intended and seem to break plugins somehow. Not sure why that is but I don't use a captcha now. Alas it's probably an upstream issue then.

fflorent commented 1 year ago

It's neither a problem here nor of the upstream probably, but of the plugins themselves.

The author of PeerTube proposed a plugin which queries Akismet in order to prevent spam (although Akismet is not a Free Software): https://framagit.org/framasoft/peertube/official-plugins/-/tree/master/peertube-plugin-akismet

Anyway, I suggest you to close the issue.

yodahome commented 1 year ago

Well, I would argue that ideally installing even a faulty plugin should not cause the whole web interface to bork but that's the main app. I used the captcha during registration, Akismet does not effectively keep scripts/bots from registering numerous accounts (and apparently checking for valid email does not help either because bots can do that too). Issue closed.