electron-userland / electron-builder

A complete solution to package and build a ready for distribution Electron app with “auto update” support out of the box
https://www.electron.build
MIT License
13.68k stars 1.74k forks source link

ASAR doesn't contain Serialport's dependency modules on Linux/Mac (works fine on Windows) #7451

Closed petervanderwalt closed 1 year ago

petervanderwalt commented 1 year ago

Updated existing project to latest Electron, and latest prerelease Electron-builder Project Builds fine - using Github Actions CI (windows-latest, ubuntu-latest and macos-latest runners)

Installed Windows version works fine, but Mac and Linux - app doesn't start, can't find depencies' submodules

https://openbuilds.com/threads/control-in-macos-linux-window-doesnt-open.20134/ relates

Require stack:

- /Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js

- /Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/node_modules/serialport/dist/serialport.js

- /Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/node_modules/serialport/dist/index.js

- /Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/index.js

-

at Module._resolveFilename (node:internal/modules/cjs/loader:1002:15)

at n._resolveFilename (node:electron/js2c/browser_init:2:109797)

at Module._load (node:internal/modules/cjs/loader:848:27)

at f._load (node:electron/js2c/asar_bundle:2:13330)

at Module.require (node:internal/modules/cjs/loader:1068:19)

at require (node:internal/modules/cjs/helpers:103:18)

at Object.<anonymous> (/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js:22:33)

at Module._compile (node:internal/modules/cjs/loader:1174:14)

at Module._extensions..js (node:internal/modules/cjs/loader:1229:10)

at Module.load (node:internal/modules/cjs/loader:1044:32) {

code: 'MODULE_NOT_FOUND',

requireStack: [

'/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js',

'/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/node_modules/serialport/dist/serialport.js',

'/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/node_modules/serialport/dist/index.js',

'/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/index.js',

undefined

]

}

TypeError: Cannot read properties of undefined (reading 'address')

at Server.<anonymous> (/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/index.js:99:41)

at Object.onceWrapper (node:events:627:28)

at Server.emit (node:events:525:35)

at emitListeningNT (node:net:1519:10)

at process.processTicksAndRejections (node:internal/process/task_queues:81:21)

TypeError: Cannot read properties of undefined (reading 'address')

at Server.<anonymous> (/Applications/OpenBuildsCONTROL.app/Contents/Resources/app.asar/index.js:103:41)

at Object.onceWrapper (node:events:627:28)

at Server.emit (node:events:525:35)

at emitListeningNT (node:net:1519:10)

at process.processTicksAndRejections (node:internal/process/task_queues:81:21)
mmaietta commented 1 year ago

Seems like a local configuration issue. What's in /Contents/Resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp or root dir?

petervanderwalt commented 1 year ago

Was working fine with "electron": "^19.0.17", and "electron-builder": "^23.3.3", ONLY change to cause it to break for Mac/Linux is to update to "electron": "^23.1.1" and "electron-builder": "^v24.0.0-alpha.13",

This config works fine on Windows (making it less likely to be config, or I'd expect all three to fail)

The .node file etc is inside the directory, but inside /Contents/Resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp node_modules doesn't have the same depency directories that are present on Windows - can post screenshots later just not at the Mac machine at the moment

petervanderwalt commented 1 year ago

@develar - I tracked it down to electron-builder v24.0.0-alpha.13

Could you check my local config just to make sure its not something I need to change to adapt to newest builder? I can't use electron-builder 23 because it has bugs with building on Github Actions for Windows.

Package.json https://github.com/OpenBuilds/OpenBuilds-CONTROL/blob/bf2c2a8c1c2ae95798a08053d8babbPace23e792066/package.json

Succesfully builds on Github Actions https://github.com/OpenBuilds/OpenBuilds-CONTROL/actions/runs/4305843127

Resultant Mac and Linux asars are missing submodules

(Can't use Electron 23, or the Windows compiles fail: https://github.com/OpenBuilds/OpenBuilds-CONTROL/actions/runs/4196887939 (windows and Linux worked fine) - 24.x works fine for Windows (so had to update for that), but broke mac and linux

petervanderwalt commented 1 year ago

PS: Labels are set wrong. Mac and Linux not Mac and Windows (:

mmaietta commented 1 year ago

I tracked it down to electron-builder v24.0.0-alpha.13

Did it only start in alpha.13 or does it exist in previous versions? Trying to git diff the release versions https://github.com/electron-userland/electron-builder/compare/v24.0.0-alpha.12...v24.0.0-alpha.13

Re: the issue labels. I'll see if I can override them. They currently are automated, and perhaps not accurately. (It was a test automation)

petervanderwalt commented 1 year ago

I'll go back one alpha at a time and report back.

petervanderwalt commented 1 year ago

Right good check

Test setup: 1) Spun up a fresh Ubuntu 22.10 Desktop VM, install Node 18, Yarn, Git. 2) Clone https://github.com/OpenBuilds/OpenBuilds-CONTROL 3) yarn install depencies

Test 1:

(but breaks Windows compiles, can't use it thats why I am trying to use 24)

Test 2:

Test 3=alpha11, Test 4 =alpha10, still broken - skipped down to Alpha.1 for next test to save time

Test 4:

(log output in next reply)

petervanderwalt commented 1 year ago
user@user-Virtual-Machine:~/Desktop/OpenBuilds-CONTROL$ ./node_modules/.bin/electron-builder 
  • electron-builder  version=24.0.0-alpha.1 os=5.19.0-21-generic
  • loaded configuration  file=package.json ("build" field)
  • electron-rebuild is already incorporated into electron-builder, please consider to remove excess dependency from devDependencies

To ensure your native dependencies are always matched electron version, simply add script `"postinstall": "electron-builder install-app-deps" to your `package.json`
  • writing effective config  file=dist/builder-effective-config.yaml
  • executing electron-rebuild  appDir=/home/user/Desktop/OpenBuilds-CONTROL arch=x64
  • loaded configuration  file=package.json ("build" field)
make: Entering directory '/home/user/Desktop/OpenBuilds-CONTROL/node_modules/@serialport/bindings/build'
  CXX(target) Release/obj.target/bindings/src/serialport.o
In file included from /home/user/.electron-gyp/23.1.2/include/node/v8-object.h:10,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8-array-buffer.h:13,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8.h:24,
                 from /home/user/.electron-gyp/23.1.2/include/node/node.h:79,
                 from ../../../nan/nan.h:60,
                 from ../src/./serialport.h:13,
                 from ../src/serialport.cpp:1:
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:256:44: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  256 |   using CopyablePersistent = Persistent<T, CopyablePersistentTraits<T>>;
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
../../../nan/nan.h:206:16: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  206 |     public v8::CopyablePersistentTraits<T> {};
      |                ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Open(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:78:51: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
   78 |   uv_queue_work(uv_default_loop(), req, EIO_Open, (uv_after_work_cb)EIO_AfterOpen);
      |                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Update(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:135:53: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  135 |   uv_queue_work(uv_default_loop(), req, EIO_Update, (uv_after_work_cb)EIO_AfterUpdate);
      |                                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Close(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:175:52: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  175 |   uv_queue_work(uv_default_loop(), req, EIO_Close, (uv_after_work_cb)EIO_AfterClose);
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Flush(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:215:52: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  215 |   uv_queue_work(uv_default_loop(), req, EIO_Flush, (uv_after_work_cb)EIO_AfterFlush);
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Set(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:271:50: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  271 |   uv_queue_work(uv_default_loop(), req, EIO_Set, (uv_after_work_cb)EIO_AfterSet);
      |                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Get(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:316:50: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  316 |   uv_queue_work(uv_default_loop(), req, EIO_Get, (uv_after_work_cb)EIO_AfterGet);
      |                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE GetBaudRate(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:366:58: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  366 |   uv_queue_work(uv_default_loop(), req, EIO_GetBaudRate, (uv_after_work_cb)EIO_AfterGetBaudRate);
      |                                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: In function ‘Nan::NAN_METHOD_RETURN_TYPE Drain(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/serialport.cpp:412:52: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
  412 |   uv_queue_work(uv_default_loop(), req, EIO_Drain, (uv_after_work_cb)EIO_AfterDrain);
      |                                                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp: At global scope:
../src/serialport.cpp:433:28: warning: unnecessary parentheses in declaration of ‘ToParityEnum’ [-Wparentheses]
  433 | SerialPortParity NAN_INLINE(ToParityEnum(const v8::Local<v8::String>& v8str)) {
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp:433:28: note: remove parentheses
  433 | SerialPortParity NAN_INLINE(ToParityEnum(const v8::Local<v8::String>& v8str)) {
      |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                            -                                                -
../src/serialport.cpp:452:30: warning: unnecessary parentheses in declaration of ‘ToStopBitEnum’ [-Wparentheses]
  452 | SerialPortStopBits NAN_INLINE(ToStopBitEnum(double stopBits)) {
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport.cpp:452:30: note: remove parentheses
  452 | SerialPortStopBits NAN_INLINE(ToStopBitEnum(double stopBits)) {
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                              -                              -
/home/user/.electron-gyp/23.1.2/include/node/node.h:991:7: warning: cast between incompatible function types from ‘void (*)(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)’ {aka ‘void (*)(v8::Local<v8::Object>)’} to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
  991 |       (node::addon_register_func) (regfunc),                          \
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/node.h:1025:3: note: in expansion of macro ‘NODE_MODULE_X’
 1025 |   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
      |   ^~~~~~~~~~~~~
../src/serialport.cpp:486:1: note: in expansion of macro ‘NODE_MODULE’
  486 | NODE_MODULE(serialport, init);
      | ^~~~~~~~~~~
  CXX(target) Release/obj.target/bindings/src/serialport_unix.o
In file included from /home/user/.electron-gyp/23.1.2/include/node/v8-object.h:10,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8-array-buffer.h:13,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8.h:24,
                 from /home/user/.electron-gyp/23.1.2/include/node/node.h:79,
                 from ../../../nan/nan.h:60,
                 from ../src/serialport.h:13,
                 from ../src/serialport_unix.cpp:2:
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:256:44: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  256 |   using CopyablePersistent = Persistent<T, CopyablePersistentTraits<T>>;
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
../../../nan/nan.h:206:16: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  206 |     public v8::CopyablePersistentTraits<T> {};
      |                ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
../src/serialport_unix.cpp: In function ‘int setup(int, OpenBaton*)’:
../src/serialport_unix.cpp:179:82: warning: ‘%s’ directive output may be truncated writing up to 1023 bytes into a region of size 1005 [-Wformat-truncation=]
  179 |     snprintf(data->errorString, sizeof(data->errorString), "Error %s Cannot open %s", strerror(errno), data->path);
      |                                                                                  ^~
In file included from /usr/include/stdio.h:906,
                 from ../src/serialport.h:10:
In function ‘int snprintf(char*, size_t, const char*, ...)’,
    inlined from ‘int setup(int, OpenBaton*)’ at ../src/serialport_unix.cpp:179:13:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:54:35: note: ‘__builtin___snprintf_chk’ output 20 or more bytes (assuming 1043) into a destination of size 1024
   54 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   55 |                                    __glibc_objsize (__s), __fmt,
      |                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   56 |                                    __va_arg_pack ());
      |                                    ~~~~~~~~~~~~~~~~~
../src/serialport_unix.cpp: In function ‘void EIO_Open(uv_work_t*)’:
../src/serialport_unix.cpp:89:84: warning: ‘%s’ directive output may be truncated writing up to 1023 bytes into a region of size 1003 [-Wformat-truncation=]
   89 |     snprintf(data->errorString, sizeof(data->errorString), "Error: %s, cannot open %s", strerror(errno), data->path);
      |                                                                                    ^~
In function ‘int snprintf(char*, size_t, const char*, ...)’,
    inlined from ‘void EIO_Open(uv_work_t*)’ at ../src/serialport_unix.cpp:89:13:
/usr/include/x86_64-linux-gnu/bits/stdio2.h:54:35: note: ‘__builtin___snprintf_chk’ output 22 or more bytes (assuming 1045) into a destination of size 1024
   54 |   return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1,
      |          ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   55 |                                    __glibc_objsize (__s), __fmt,
      |                                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   56 |                                    __va_arg_pack ());
      |                                    ~~~~~~~~~~~~~~~~~
  CXX(target) Release/obj.target/bindings/src/poller.o
In file included from /home/user/.electron-gyp/23.1.2/include/node/v8-object.h:10,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8-array-buffer.h:13,
                 from /home/user/.electron-gyp/23.1.2/include/node/v8.h:24,
                 from /home/user/.electron-gyp/23.1.2/include/node/node.h:79,
                 from ../../../nan/nan.h:60,
                 from ../src/poller.cpp:1:
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:256:44: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  256 |   using CopyablePersistent = Persistent<T, CopyablePersistentTraits<T>>;
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
../../../nan/nan.h:206:16: warning: ‘template<class T> struct v8::CopyablePersistentTraits’ is deprecated: Use v8::Global instead [-Wdeprecated-declarations]
  206 |     public v8::CopyablePersistentTraits<T> {};
      |                ^~~~~~~~~~~~~~~~~~~~~~~~
/home/user/.electron-gyp/23.1.2/include/node/v8-persistent-handle.h:255:48: note: declared here
  255 | struct V8_DEPRECATED("Use v8::Global instead") CopyablePersistentTraits {
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~
  CXX(target) Release/obj.target/bindings/src/serialport_linux.o
  SOLINK_MODULE(target) Release/obj.target/bindings.node
  COPY Release/bindings.node
make: Leaving directory '/home/user/Desktop/OpenBuilds-CONTROL/node_modules/@serialport/bindings/build'
make: Entering directory '/home/user/Desktop/OpenBuilds-CONTROL/node_modules/@serialport/bindings-cpp/build'
  CXX(target) Release/obj.target/bindings/src/serialport.o
  CXX(target) Release/obj.target/bindings/src/serialport_unix.o
  CXX(target) Release/obj.target/bindings/src/poller.o
  CXX(target) Release/obj.target/bindings/src/serialport_linux.o
  SOLINK_MODULE(target) Release/obj.target/bindings.node
  COPY Release/bindings.node
make: Leaving directory '/home/user/Desktop/OpenBuilds-CONTROL/node_modules/@serialport/bindings-cpp/build'
  • packaging       platform=linux arch=x64 electron=23.1.2 appOutDir=dist/linux-unpacked
  • building        target=AppImage arch=x64 file=dist/OpenBuildsCONTROL-1.0.356.AppImage
  • building        target=deb arch=x64 file=dist/OpenBuildsCONTROL_1.0.356_amd64.deb
user@user-Virtual-Machine:~/Desktop/OpenBuilds-CONTROL$ dpkg -i dist/OpenBuildsCONTROL_1.0.356_amd64.deb 
dpkg: error: requested operation requires superuser privilege
user@user-Virtual-Machine:~/Desktop/OpenBuilds-CONTROL$ sudo dpkg -i dist/OpenBuildsCONTROL_1.0.356_amd64.deb 
[sudo] password for user: 
(Reading database ... 172239 files and directories currently installed.)
Preparing to unpack .../OpenBuildsCONTROL_1.0.356_amd64.deb ...
Unpacking openbuildscontrol (1.0.356) over (1.0.356) ...
Setting up openbuildscontrol (1.0.356) ...
Processing triggers for mailcap (3.70+nmu1ubuntu1) ...
Processing triggers for gnome-menus (3.36.0-1ubuntu3) ...
Processing triggers for desktop-file-utils (0.26-1ubuntu4) ...
Processing triggers for hicolor-icon-theme (0.17-2) ...
user@user-Virtual-Machine:~/Desktop/OpenBuilds-CONTROL$ /opt/OpenBuildsCONTROL/openbuildscontrol 
Error: Cannot find module 'debug'
Require stack:
- /opt/OpenBuildsCONTROL/resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js
- /opt/OpenBuildsCONTROL/resources/app.asar/node_modules/serialport/dist/serialport.js
- /opt/OpenBuildsCONTROL/resources/app.asar/node_modules/serialport/dist/index.js
- /opt/OpenBuildsCONTROL/resources/app.asar/index.js
- 
    at Module._resolveFilename (node:internal/modules/cjs/loader:1002:15)
    at n._resolveFilename (node:electron/js2c/browser_init:2:109797)
    at Module._load (node:internal/modules/cjs/loader:848:27)
    at f._load (node:electron/js2c/asar_bundle:2:13330)
    at Module.require (node:internal/modules/cjs/loader:1068:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/opt/OpenBuildsCONTROL/resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js:22:33)
    at Module._compile (node:internal/modules/cjs/loader:1174:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1229:10)
    at Module.load (node:internal/modules/cjs/loader:1044:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/opt/OpenBuildsCONTROL/resources/app.asar.unpacked/node_modules/@serialport/bindings-cpp/dist/index.js',
    '/opt/OpenBuildsCONTROL/resources/app.asar/node_modules/serialport/dist/serialport.js',
    '/opt/OpenBuildsCONTROL/resources/app.asar/node_modules/serialport/dist/index.js',
    '/opt/OpenBuildsCONTROL/resources/app.asar/index.js',
    undefined
  ]
}
TypeError: Cannot read properties of undefined (reading 'address')
    at Server.<anonymous> (/opt/OpenBuildsCONTROL/resources/app.asar/index.js:111:41)
    at Object.onceWrapper (node:events:627:28)
    at Server.emit (node:events:525:35)
    at emitListeningNT (node:net:1519:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)
TypeError: Cannot read properties of undefined (reading 'address')
    at Server.<anonymous> (/opt/OpenBuildsCONTROL/resources/app.asar/index.js:115:41)
    at Object.onceWrapper (node:events:627:28)
    at Server.emit (node:events:525:35)
    at emitListeningNT (node:net:1519:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)
[13274:0303/184005.955159:ERROR:viz_main_impl.cc(186)] Exiting GPU process due to errors during initialization
[13295:0303/184006.038381:ERROR:gpu_memory_buffer_support_x11.cc(49)] dri3 extension not supported.
^Cuser@user-Virtual-Machine:~/Desktop/OpenBuilds-CONTROL$ 
mmaietta commented 1 year ago

Hmmm, this is the diff https://github.com/electron-userland/electron-builder/compare/v23.6.0...v24.0.0-alpha.1

I'm thinking this may be due to this change? https://github.com/electron-userland/electron-builder/commit/5616f23ce3d03a4e71c7b7bd515ec958b1631b8b

We had to migrate to electron/rebuild in order to support newer versions of electron. Curious if the .node file is not being generated at all? I'm not seeing any errors in the build logs though.

petervanderwalt commented 1 year ago

We had to migrate to electron/rebuild in order to support newer versions of electron.

For a moment I thought this might be related - I still had "electron-rebuild": "^3.2.9", as devDependency (when running electron in dev, before building with electron-builder). Figured ah! Maybe the old version is messing with the later build. Removed electron-rebuild completely and ran electron-builder again, same issue - missing dependencies in the final install.

Curious if the .node file is not being generated at all?

It is generated and packed into the ASAR (unpacked the asar to confirm)

image

petervanderwalt commented 1 year ago

On windows (Where is all works), the unpacked asar does not have a serialport/binding-cpp/bin directory. It looks to be using the prebuilds/win32-xx/node.napi.node instead?

@reconbot @serialport

mmaietta commented 1 year ago

Hmmm, woah, the .node file shouldn't be in the asar AFAIK, I always thought those had to be in the asar-unpacked folder. Can you try forcing the .node to be included in asarUnpack?

Side note, electron/notarize is automatically included as a build step in v24 with the correct Env Vars specified, so this additional "afterSign": "electron-builder-notarize", will no longer be necessary 🙂 https://www.electron.build/configuration/mac

APPLE_ID and APPLE_APP_SPECIFIC_PASSWORD via environment variables to activate notarization step

petervanderwalt commented 1 year ago

Hmmm, woah, the .node file shouldn't be in the asar AFAIK, I always thought those had to be in the asar-unpacked folder. Can you try forcing the .node to be included in asarUnpack?

Included @serialport in asarUnpack

image

But still same issue.

Side note, electron/notarize is automatically included as a build step in v24 with the correct Env Vars specified, so this additional "afterSign": "electron-builder-notarize", will no longer be necessary 🙂 https://www.electron.build/configuration/mac

Thanks! Will remove it as soon as we can stick with V24! (:

petervanderwalt commented 1 year ago

Going nuclear and asarUnpack the entire node_modules directory - does bring it back to life - reaffirming the theory that V24 does something different with identifying/packing the ASAR

I can't use the nuclear option on Mac/Windows though - as CI fails because it find other executables (python etc) and cannot codesign those as they are already signed (works fine when not asarUnpacking the whole node modules) - will Post CI logs shortly showing that scenario

petervanderwalt commented 1 year ago

https://github.com/OpenBuilds/OpenBuilds-CONTROL/actions/runs/4347892250/jobs/7595754024#step:6:269 Example run with the nuclear option https://github.com/OpenBuilds/OpenBuilds-CONTROL/blob/908eefa46a8e527ab219b6dfd0bebc8e4390d02e/package.json#L130 (so "works" for Ubuntu, breaks windows (and not good practice, need Asar for speed)

mmaietta commented 1 year ago

My apologies, we've had previous tickets/issues opened where .node files were supposed to be signed for WDAC (https://github.com/electron-userland/electron-builder/issues/7329), is it not supposed to be the case for your .node files?

Also, I took a full read through the https://github.com/electron-userland/electron-builder/compare/v23.6.0...v24.0.0-alpha.1 diff and there are only two other changes that could potentially impact globbing. Updating read-config-file and globby dependencies. Can you try pinning your globby: 11.0.4?

Alternatively, if you could provide a minimum repro repo for this, I can check it out locally when I have time

petervanderwalt commented 1 year ago

I will right away get to work on getting you a minimum repo with CI (and give you access to the repo) so you can properly help me check it out (:

Appreciate the help!

My apologies, we've had previous tickets/issues opened where .node files were supposed to be signed for WDAC (#7329), is it not supposed to be the case for your .node files?

Didn't dig that deep, all I knew was with node_modules not included in asarUnpack the CI build for windows succeeds, with it unpacked it fails on that step as linked.

petervanderwalt commented 1 year ago

I dug into the minimal example - and in doing so stripped away all but the necessities - and low and behold - it all works. So then I started adding back stuff, to get it to break again

It was added to our project a couple years ago to help with resolving esptool python scripts https://github.com/espressif/esptool that we spawn https://github.com/OpenBuilds/OpenBuilds-CONTROL/blob/b7664371d916448255e34c214a6ad5e7f8a66a90/index.js#L3350-L3355 to flash firmware onto our devices.

So in itself, not the issue (as its been there for years) but does make sense, as it changes behaviour of built-in FS functionality. Curious as to "why" it only rears issues with Electron-builder 24, and works fine with 23.6 - but for us - its a huge relief as we found root cause. Not going to dig deeper, Hazardous hasn't been updated in forever

I can move forward for now, removing Hazardous, making sure our ESPtool Spawns work correctly without it etc, but at least! We are a long way ahead of the darkness we were in for a couple weeks now. Thank you SOOOO much for helping us work through it. Its always tricky when something has been there for years, and something unrelated gets updated - it's hard to find the real blame sometimes.