search

GitSquared / edex-ui

A cross-platform, customizable science fiction terminal emulator with advanced monitoring & touchscreen support.
GNU General Public License v3.0
40.49k stars 2.55k forks source link

Fix CI pipeline for armv7l target #1051

Closed Itai-Nelken closed 3 years ago

Itai-Nelken commented 3 years ago

Technical information

Using version:

Running on:

How comfortable you are with your system and/or IT in general:

Problem

when running the latest (2.2.6) armv7l appimage on my raspberry pi 4 4gb I get the following errors:

▶  start     Starting eDEX-UI v2.2.6
ℹ  info      With Node 12.18.3 and Electron 11.3.0
ℹ  info      Renderer is Chrome 87.0.4280.141
▶  Startup   Initialized timer...
ℹ  info      Base config dir is /home/pi/.config/eDEX-UI
ℹ  info      Default settings written to /home/pi/.config/eDEX-UI/settings.json
ℹ  info      Default keymap written to /home/pi/.config/eDEX-UI/shortcuts.json
ℹ  info      Default last window state written to /home/pi/.config/eDEX-UI/lastWindowState.json
☐  pending   Mirroring internal assets...
☐  pending   Loading settings file...
☐  pending   Resolving shell path...
ℹ  info      Shell found at /usr/bin/bash
✔  success   Settings loaded!
☐  pending   Creating new terminal process on port 3000
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js
- /tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/index.js
- /tmp/.mount_eDEX-Uw94BFX/resources/app.asar/classes/terminal.class.js
- /tmp/.mount_eDEX-Uw94BFX/resources/app.asar/_boot.js
- 
    at Module._resolveFilename (internal/modules/cjs/loader.js:972:15)
    at Function.n._resolveFilename (electron/js2c/browser_init.js:249:921)
    at Module._load (internal/modules/cjs/loader.js:848:27)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:31:15)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1173:10)
    at Module.load (internal/modules/cjs/loader.js:992:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js',
    '/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/index.js',
    '/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/classes/terminal.class.js',
    '/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/_boot.js',
    undefined
  ]
}
(node:5729) UnhandledPromiseRejectionWarning: Error: /tmp/.org.chromium.Chromium.aQWTcq: wrong ELF class: ELFCLASS64
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1812)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1203:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:2039)
    at Module.load (internal/modules/cjs/loader.js:992:32)
    at Module._load (internal/modules/cjs/loader.js:885:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-Uw94BFX/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:27:11)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
(node:5729) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:5729) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Botspot commented 3 years ago

I have the same issue as well.

GitSquared commented 3 years ago

Thanks to @Botspot I was able to troubleshoot this directly on a Pi.

It seems that our CI build process doesn't target armv7 correctly for some reason. I've produced a working AppImage build for armv7, and uploaded it in place of the broken one for the v2.2.6 release.

Will look into repairing the automated builds.

GitSquared commented 3 years ago

Could anyone test this build on armv7? Tried building it with an AWS arm instance.

https://home.gaby.dev:5001/sharing/iVl7axE2V

Botspot commented 3 years ago

Could anyone test this build on armv7? Tried building it with an AWS arm instance.

https://home.gaby.dev:5001/sharing/iVl7axE2V

Looks like the same error as before.

pi@raspberrypi:~ $ /home/pi/Downloads/eDEX-UI-Linux-armv7l.AppImage
▶  start     Starting eDEX-UI v2.2.6
ℹ  info      With Node 12.18.3 and Electron 11.3.0
ℹ  info      Renderer is Chrome 87.0.4280.141
▶  Startup   Initialized timer...
ℹ  info      Base config dir is /home/pi/.config/eDEX-UI
☐  pending   Mirroring internal assets...
☐  pending   Loading settings file...
☐  pending   Resolving shell path...
ℹ  info      Shell found at /bin/bash
✔  success   Settings loaded!
☐  pending   Creating new terminal process on port 3000
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js
- /tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/index.js
- /tmp/.mount_eDEX-U31boHB/resources/app.asar/classes/terminal.class.js
- /tmp/.mount_eDEX-U31boHB/resources/app.asar/_boot.js
- 
    at Module._resolveFilename (internal/modules/cjs/loader.js:972:15)
    at Function.n._resolveFilename (electron/js2c/browser_init.js:249:921)
    at Module._load (internal/modules/cjs/loader.js:848:27)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:31:15)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1173:10)
    at Module.load (internal/modules/cjs/loader.js:992:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js',
    '/tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/index.js',
    '/tmp/.mount_eDEX-U31boHB/resources/app.asar/classes/terminal.class.js',
    '/tmp/.mount_eDEX-U31boHB/resources/app.asar/_boot.js',
    undefined
  ]
}
(node:5145) UnhandledPromiseRejectionWarning: Error: /tmp/.org.chromium.Chromium.HJK4QQ: wrong ELF class: ELFCLASS64
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1812)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1203:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:2039)
    at Module.load (internal/modules/cjs/loader.js:992:32)
    at Module._load (internal/modules/cjs/loader.js:885:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-U31boHB/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:27:11)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
(node:5145) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:5145) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
GitSquared commented 3 years ago

Crap. I need to get access to a x32 armv7 machine then, x64 armv8 doesn't cut it. Thanks for testing.

Botspot commented 3 years ago

Crap. I need to get access to a x32 armv7 machine then, x64 armv8 doesn't cut it. Thanks for testing.

Raspberry Pies are 64-bit armv8 devices, but they run a 32-bit kernel. Maybe your build machine would work if you were running it also on a 32-bit kernel?

GitSquared commented 3 years ago

Thanks for the insights... I'd eventually like to automate it so I need to find somewhere I can reliably create instances with x32 kernels, not sure how.

In the meantime I may have found a way to force gcc to build native deps in x32, can you give it a go? https://home.gaby.dev:5001/sharing/Y94HvfyTO

(thanks for your help through this, by the way - really appreciate it!)

Botspot commented 3 years ago

Thanks for the insights... I'd eventually like to automate it so I need to find somewhere I can reliably create instances with x32 kernels, not sure how.

In the meantime I may have found a way to force gcc to build native deps in x32, can you give it a go? https://home.gaby.dev:5001/sharing/Y94HvfyTO

(thanks for your help through this, by the way - really appreciate it!)

Nope, same error.

pi@raspberrypi:~ $ /home/pi/Downloads/eDEX-UI-Linux-armv7l.AppImage
▶  start     Starting eDEX-UI v2.2.6
ℹ  info      With Node 12.18.3 and Electron 11.3.0
ℹ  info      Renderer is Chrome 87.0.4280.141
▶  Startup   Initialized timer...
ℹ  info      Base config dir is /home/pi/.config/eDEX-UI
☐  pending   Mirroring internal assets...
☐  pending   Loading settings file...
☐  pending   Resolving shell path...
ℹ  info      Shell found at /bin/bash
✔  success   Settings loaded!
☐  pending   Creating new terminal process on port 3000
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js
- /tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/index.js
- /tmp/.mount_eDEX-UeBlSAx/resources/app.asar/classes/terminal.class.js
- /tmp/.mount_eDEX-UeBlSAx/resources/app.asar/_boot.js
- 
    at Module._resolveFilename (internal/modules/cjs/loader.js:972:15)
    at Function.n._resolveFilename (electron/js2c/browser_init.js:249:921)
    at Module._load (internal/modules/cjs/loader.js:848:27)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:31:15)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1173:10)
    at Module.load (internal/modules/cjs/loader.js:992:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js',
    '/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/index.js',
    '/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/classes/terminal.class.js',
    '/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/_boot.js',
    undefined
  ]
}
(node:32044) UnhandledPromiseRejectionWarning: Error: /tmp/.org.chromium.Chromium.FX1o5N: wrong ELF class: ELFCLASS64
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1812)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1203:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:2039)
    at Module.load (internal/modules/cjs/loader.js:992:32)
    at Module._load (internal/modules/cjs/loader.js:885:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-UeBlSAx/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:27:11)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
(node:32044) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:32044) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Botspot commented 3 years ago

Would it be possible for you to send me a list of terminal commands (a bash script), that would build an eDEX-UI AppImage on my machine?

I could report back error outputs and such.

GitSquared commented 3 years ago

@Botspot Sure, these are the steps to build only the armv7l target:

(check that you have node >=14 and npm >=7)

git clone https://github.com/GitSquared/edex-ui.git
cd edex-ui
npm install
npm run prebuild-linux
npx electron-builder -l --armv7l

Note that the prebuild step can take a while, esp. on a low-power device such as a Pi. AppImage and unpacked build outputs will be in dist/.

Itai-Nelken commented 3 years ago

this problem also affects the arm64 builds:

▶  start     Starting eDEX-UI v2.2.6
ℹ  info      With Node 12.18.3 and Electron 11.3.0
ℹ  info      Renderer is Chrome 87.0.4280.141
▶  Startup   Initialized timer...
ℹ  info      Base config dir is /home/itai/.config/eDEX-UI
ℹ  info      Default settings written to /home/itai/.config/eDEX-UI/settings.json
ℹ  info      Default keymap written to /home/itai/.config/eDEX-UI/shortcuts.json
ℹ  info      Default last window state written to /home/itai/.config/eDEX-UI/lastWindowState.json
☐  pending   Mirroring internal assets...
☐  pending   Loading settings file...
☐  pending   Resolving shell path...
ℹ  info      Shell found at /bin/bash
✔  success   Settings loaded!
☐  pending   Creating new terminal process on port 3000
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js
- /tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/index.js
- /tmp/.mount_eDEX-UV6rEuo/resources/app.asar/classes/terminal.class.js
- /tmp/.mount_eDEX-UV6rEuo/resources/app.asar/_boot.js
- 
    at Module._resolveFilename (internal/modules/cjs/loader.js:972:15)
    at Function.n._resolveFilename (electron/js2c/browser_init.js:249:921)
    at Module._load (internal/modules/cjs/loader.js:848:27)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:31:15)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1173:10)                                                                           
    at Module.load (internal/modules/cjs/loader.js:992:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js',
    '/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/index.js',
    '/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/classes/terminal.class.js',
    '/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/_boot.js',
    undefined
  ]
}
(node:120388) UnhandledPromiseRejectionWarning: Error: /tmp/.org.chromium.Chromium.qftvkm: cannot open shared object file: No such file or directory
    at process.func [as dlopen] (electron/js2c/asar_bundle.js:5:1812)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1203:18)
    at Object.func [as .node] (electron/js2c/asar_bundle.js:5:2039)
    at Module.load (internal/modules/cjs/loader.js:992:32)
    at Module._load (internal/modules/cjs/loader.js:885:14)
    at Function.f._load (electron/js2c/asar_bundle.js:5:12738)
    at Module.require (internal/modules/cjs/loader.js:1032:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/tmp/.mount_eDEX-UV6rEuo/resources/app.asar/node_modules/node-pty/lib/unixTerminal.js:27:11)
    at Module._compile (internal/modules/cjs/loader.js:1152:30)
(node:120388) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:120388) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
GitSquared commented 3 years ago

@Itai-Nelken Can you give me the uname -a on your machine?

Itai-Nelken commented 3 years ago

@Itai-Nelken Can you give me the uname -a on your machine?

I'm on manjaro ARM kde plasma right now: Linux manjaro-pi4 5.10.17-1-MANJARO-ARM #1 SMP PREEMPT Mon Feb 22 11:29:03 CST 2021 aarch64 GNU/Linux

Itai-Nelken commented 3 years ago

@GitSquared this might help: https://github.com/flameshot-org/flameshot/pull/1460 they implemented building on arm with docker and qemu if I understood correctly.

GitSquared commented 3 years ago

Thanks for the tip. Using qemu seems like a good idea indeed... I've tried a few things on a new branch but I can't get Actions to run some steps in a multiarch container. Will keep this thread updated.

Flameshot's setup is very complicated (to me, ofc) they use a custom docker image that actually runs all the builds stuff, their Actions workflow is only responsible for spinning it up.

GitSquared commented 3 years ago

Update: I succesfully got arm64 builds to run on QEMU + Docker (artifact, if you want to try it).

However, I'm still blocked on armv7 because the QEMU helper says the machine type is unsupported for some reason:

2021/03/17 13:28:22 installing: v7 unsupported architecture: v7
{
  "supported": [
    "linux/amd64",
    "linux/386"
  ],
Run docker://arm32v7/node:14-buster
/usr/bin/docker run --name arm32v7node14buster_fd1d0b --label 5588e4 --workdir /github/workspace (...)
WARNING: The requested image's platform (linux/arm) does not match the detected host platform (linux/amd64) and no specific platform was requested
standard_init_linux.go:219: exec user process caused: exec format error

I'm not sure why arm32 emulation wouldn't be possible on the Azure instances used by Actions. Will look into it, maybe i can force-enable it.

GitSquared commented 3 years ago

Got armv7 emulation working! Binary links: armv7 | armv8. If someone can confirm these works, I'll merge my test branch and we can close this (hopefully release a new version too, won't be able to fix m1 builds anyway).

cc @Botspot @Itai-Nelken

Botspot commented 3 years ago

Got armv7 emulation working! Binary links: armv7 | armv8. If someone can confirm these works, I'll merge my test branch and we can close this (hopefully release a new version too, won't be able to fix m1 builds anyway).

cc @Botspot @Itai-Nelken

It works!!!

Not only that, but it launches in about 10 seconds. All previous appimages took over 2 minutes to load. What did you do to reduce the launch time so significantly?

As a sidenote, is there a reason it didn't open in fullscreen? Is that the new normal? (All previous eDEX-UI appimages would open in fullscreen)

Itai-Nelken commented 3 years ago

As a sidenote, is there a reason it didn't open in fullscreen? Is that the new normal? (All previous eDEX-UI appimages would open in fullscreen)

@Botspot the latest version doesn't open in full screen on my mac as well.

GitSquared commented 3 years ago

It works!!!

Finally!! :tada: :tada: Will merge it now. @Itai-Nelken thanks a lot for linking Flameshot's builds, I would never have thought of using qemu and docker.

Not only that, but it launches in about 10 seconds. All previous appimages took over 2 minutes to load. What did you do to reduce the launch time so significantly?

Wow, that's really great, I'm not sure how much I can take credit for this. I have done some optimizations on source code minification recently which might have had an effect here.

Re: fullscreen, interesting, as usual could you guys check the forceFullscreen allowWindowed settings? Is there any open issues on this already? I need to take another look

GitSquared commented 3 years ago

Released v2.2.7.

Botspot commented 3 years ago

Re: fullscreen, interesting, as usual could you guys check the forceFullscreen allowWindowed settings? Is there any open issues on this already? I need to take another look

What did you want me to check?

GitSquared commented 3 years ago

Whether the "forceFullscreen" and/or "allowWindowed" edex settings affected the issue. I may make a new thread if I can reproduce it

Itai-Nelken commented 3 years ago

@GitSquared on TwisterOS it does work in fullscreen.

Botspot commented 3 years ago

I realized what happened: last time I used eDEX-UI, I configured it to launch not in full screen.

Well apparently the new appimage remembered the settings. But I didn't remember.

GitSquared commented 3 years ago

Config is stored in ~/.config/eDEX-UI 😉

Botspot commented 3 years ago

Config is stored in ~/.config/eDEX-UI 😉

Does edex-ui use its own Chromium rendering engine, or the system-installed one?

Botspot commented 3 years ago

@GitSquared ^

GitSquared commented 3 years ago

Does edex-ui use its own Chromium rendering engine, or the system-installed one?

Edex comes bundled with Electron which includes the Chromium engine.

Botspot commented 3 years ago

Does edex-ui use its own Chromium rendering engine, or the system-installed one?

Edex comes bundled with Electron which includes the Chromium engine.

Would there be any possible way to use the system installed Chromium version?

Here's why I ask that:
The Raspberry Pi Foundation pays several developers full-time to optimize Chromium. As a result, it renders everything much faster than ordinary standard Chromium or Electron. They've fully GPU-accelerated everything so YouTube, WebGL, and animations run about ten times smoother.

I think that if eDEX-UI used this optimized version of Chromium, it would run with a much higher framerate and minimal CPU usage. (Currently the globe is about 8-10fps and eDEX-UI uses 100% CPU constantly)

GitSquared commented 3 years ago

Edex doesn't use Chromium directly though, so we need to find an Electron version that works with both their own Chromium patches and rpi ones.

Electron is quite popular, has this been attempted previously on other apps that use it? Like VS Code / Discord / Slack etc...

Botspot commented 3 years ago

Edex doesn't use Chromium directly though, so we need to find an Electron version that works with both their own Chromium patches and rpi ones.

Electron is quite popular, has this been attempted previously on other apps that use it? Like VS Code / Discord / Slack etc...

Okay I'd like to try changing some flags in electron.
Is this sufficient? (providing flags directly to the appimage)

~/eDEX-UI-RPi/eDEX-UI-Linux-armv7l_2.2.7.AppImage --use-gl=egl --force-renderer-accessibility --disable-quic --enable-tcp-fast-open
GitSquared commented 3 years ago

Unfortunately no, you can't pass flags through the AppImage container.

If you find some Electron/Chromium/Node.js flags that do significantly improve arm performance, I could add them to edex's code directly.

You can test flags by cloning the repo and running the following commands (example flags included):

npm run install-linux
npx electron src --use-gl=egl --force-renderer-accessibility --disable-quic --enable-tcp-fast-open --enable-logging --v=2
Botspot commented 3 years ago

You can test flags by cloning the repo and running the following commands (example flags included):

npm run install-linux
npx electron src --use-gl=egl --force-renderer-accessibility --disable-quic --enable-tcp-fast-open --enable-logging --v=2

I tried a variety of flags and none made much of a difference. Is it possible to enable a FPS counter for the globe?
Also I'm not convinced Electron is hardware accelerated. Is there a way to check that?
Another option: running Electron as a hardware overlay would certainly improve things (due to circumventing inefficient Xorg), but I have no clue how to do that, or if it's possible.

GitSquared commented 3 years ago

You can use the Performance tab in chromium's dev tools (open with ctrl+shift+i). There's an FPS graph and a GPU one that will confirm whether hardware acceleration is being used.

You can activate some performance overlays with the Rendering options, see menu screenshot (3-dots, more tools -> rendering).

running Electron as a hardware overlay

Not sure what you mean by that?

Botspot commented 3 years ago

This seems to indicate to me that Electron is not hardware accelerated. 2021-03-23-090543_1920x1080_scrot For comparison, I went to Raspberry Pi's GPU-accelerated Chromium, and turned on the same overlay on github.com: 2021-03-23-091149_1920x1080_scrot To me it looks like Electron is not HW accelerated.

running Electron as a hardware overlay

Not sure what you mean by that?

Linux devices use a Xorg server. This is used to display all windows in a desktop as we know it. Unfortunately, X is very inefficient for high-FPS applications. If you bypass the X server and display the video on top, that's called a hardware overlay.
On Raspberry Pi's custom VLC build, you can get roughly four times the performance when in fullscreen, than when playing inside a window. (thanks to their hardware overlay that's activated when in fullscreen)
That's because when you're playing the video in a window inside the X server, there's a lot of data conversions in the display stack, between the program and your screen. Playing the video directly to the screen, skipping the X server, simplifies the data path, thus boosting performance significantly.

As eDEX-UI is already in fullscreen, users wouldn't really notice if it was outside the X server or not - it wouldn't make a difference.

GitSquared commented 3 years ago

This seems to indicate to me that Electron is not hardware accelerated.

Indeed, looks like the WebGL/3D stuff is using the GPU but rasterization is done on the CPU. Not ideal.

Thanks for the explanation of X hardware overlays. Unfortunately, this is many layers of abstractions below what I'm building here, so this really depends on being able to provide an Electron build with RPI's chromium fork embedded inside it. I searched a bit but couldn't find anything, could you point me to RPI's fork?

Botspot commented 3 years ago

Thanks for the explanation of X hardware overlays. Unfortunately, this is many layers of abstractions below what I'm building here, so this really depends on being able to provide an Electron build with RPI's chromium fork embedded inside it. I searched a bit but couldn't find anything, could you point me to RPI's fork?

I am far from being the expert here unfortunately. There seems to be very little info about it, and the closest I got to compiling RPi's custom Chromium was this question I asked.
Getting this working probably won't be easy, but if your efforts improve Electron performance for all Electron applications on RPi, I'd imagine you'd receive a ton of support from the community.

Botspot commented 3 years ago

I'll do my best to help you; that includes sending you a raspberry pi if you want it.

GitSquared commented 3 years ago

I don't think I'm the right guy to do this. I could hack a meta-fork of chromium with both RPI's and Electron's patches applied, and build Electron binaries from that, but I'm not sure how feasible it would be to switch over existing (sometimes closed-source) apps to this build - you can't simply install it system-wide and point apps to it, unfortunately. I also would not have enough incentives to maintain such a project - the fact is I don't have a use for Pi devices myself and could not guarantee security and stability on a mix-and-match of code that I do not even understand.

Hope someone figures it out.

Botspot commented 3 years ago

I don't think I'm the right guy to do this. I could hack a meta-fork of chromium with both RPI's and Electron's patches applied, and build Electron binaries from that, but I'm not sure how feasible it would be to switch over existing (sometimes closed-source) apps to this build - you can't simply install it system-wide and point apps to it, unfortunately. I also would not have enough incentives to maintain such a project - the fact is I don't have a use for Pi devices myself and could not guarantee security and stability on a mix-and-match of code that I do not even understand.

Hope someone figures it out.

I understand.
Thank you. :)