search

electron / packager

Customize and package your Electron app with OS-specific bundles (.app, .exe, etc.) via JS or CLI
https://npm.im/@electron/packager
BSD 2-Clause "Simplified" License
187 stars 19 forks source link

Packaged app with native module fails to start due to not rebuilding #844

Closed tzvetelin-vassilev closed 6 years ago

tzvetelin-vassilev commented 6 years ago

With latest versions app fails to start because missing node modules.

DEBUG=electron-packager electron-packager . --asar Packaging app for platform darwin x64 using electron v2.0.0 Wrote new app to /Users/vassilev/Projects/inkspace/Wacom Inkspace App-darwin-x64

Package is created successfully but when app is started there is exception. This exception is available only after packaging. I have added 'ansi-regex' in dependencies, but then 'code-point-at' is missing and so on for may be 7-8 modules more. Package with 11.2.0 do not have problems and missing packages. I'm using lates npm and electron. if you need my package.json I can include it.

module.js:485 Uncaught Error: Cannot find module 'ansi-regex' at Module._resolveFilename (module.js:485) at Function.Module._resolveFilename (/Applications/Wacom Inkspace App.app/Contents/Resources/electron.asar/common/reset-search-paths.js:35) at Function.Module._load (module.js:437) at Module.require (module.js:513) at require (internal/module.js:11) at Object. (/Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/strip-ansi/index.js:2) at Object. (/Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/strip-ansi/index.js:8) at Module._compile (module.js:569) at Object.Module._extensions..js (module.js:580) at Module.load (module.js:503) Module._resolveFilename @ module.js:485 Module._resolveFilename @ /Applications/Wacom Inkspace App.app/Contents/Resources/electron.asar/common/reset-search-paths.js:35 Module._load @ module.js:437 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/strip-ansi/index.js:2 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/strip-ansi/index.js:8 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/string-width/index.js:2 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/string-width/index.js:39 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/wide-align/align.js:2 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/wide-align/align.js:67 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/render-template…:2 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/render-template…:183 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/plumbing.js:3 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/plumbing.js:50 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/index.js:2 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/gauge/index.js:235 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/npmlog/log.js:3 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/npmlog/log.js:311 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/node-pre-gyp/lib/node…:15 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/node-pre-gyp/lib/node…:196 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/usb/usb.js:1 (anonymous) @ /Applications/Wacom Inkspace App.app/Contents/Resources/app.asar/node_modules/usb/usb.js:408 Module._compile @ module.js:569 Module._extensions..js @ module.js:580 Module.load @ module.js:503 tryModuleLoad @ module.js:466 Module._load @ module.js:458 Module.require @ module.js:513 require @ internal/module.js:11 (anonymous) @ bufferPsdWriter.ts:3 webpack_require @ manifest.js:55 (anonymous) @ bufferPsdWriter.ts:3 webpack_require @ manifest.js:55 module.exports.Object.defineProperty.value @ bufferPsdWriter.ts:3 __webpack_require__ @ manifest.js:55 webpackJsonpCallback @ manifest.js:26 (anonymous) @ main.js:2

welcome[bot] commented 6 years ago

👋 Thanks for opening your first issue here! If you have a question about using Electron Packager, read the support docs. If you're reporting a 🐞 bug, please make sure you include steps to reproduce it. Development and issue triage is community-driven, so please be patient and we will get back to you as soon as we can.

To help make it easier for us to investigate your issue, please follow the contributing guidelines.

malept commented 6 years ago

Thanks for filing an issue! In order to help you with your problem, we're going to need more information about it. In particular:

tzvetelin-vassilev commented 6 years ago

Output from packager:

electron-packager .. --overwrite --asar --ignore="app-scripts|test" --icon=../app-icon/mac/app.icns --out=../../out

electron-packager Electron Packager 12.0.2 +0ms electron-packager Node v8.4.0 +1ms electron-packager Host Operating system: darwin (x64) +0ms electron-packager Packager Options: {"_":[".."],"deref-symlinks":true,"derefSymlinks":true,"download":{"strictSSL":true},"prune":true,"overwrite":true,"asar":true,"ignore":"app-scripts|test","icon":"../app-icon/mac/app.icns","out":"../../out","dir":"..","protocols":[]} +0ms electron-packager Target Platforms: darwin +0ms electron-packager Target Architectures: x64 +0ms electron-packager Inferring application name from productName in /Users/vassilev/Projects/mate-desktop/package.json +0ms electron-packager Inferring appVersion from version in /Users/vassilev/Projects/mate-desktop/package.json +3ms electron-packager Inferring target Electron version from electron in /Users/vassilev/Projects/mate-desktop/package.json +10ms electron-packager Application name: Wacom Inkspace App +28ms electron-packager Target Electron version: 2.0.0 +0ms electron-packager Ignored path regular expressions: [ 'app-scripts|test', '/\.git($|/)', '/node_modules/\.bin($|/)', '\.o(bj)?$' ] +0ms electron-packager Downloading Electron with options {"strictSSL":true,"platform":"darwin","arch":"x64","version":"2.0.0"} +0ms Packaging app for platform darwin x64 using electron v2.0.0 electron-packager Creating /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64-template +2s electron-packager Extracting /Users/vassilev/.electron/electron-v2.0.0-darwin-x64.zip to /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64-template +0ms electron-packager Initializing app in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64 from /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64-template template +0ms electron-packager Ignored paths based on the out param: [ '/Users/vassilev/Projects/out' ] +4s electron-packager Running asar with the options {} +4m electron-packager Copying icon "../app-icon/mac/app.icns" to app's Resources as "electron.icns" +0ms electron-packager Renaming Electron to Wacom Inkspace App in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/MacOS +7s electron-packager Renaming Electron Helper to Wacom Inkspace App Helper in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks/Electron Helper.app/Contents/MacOS +1ms electron-packager Renaming Electron Helper EH to Wacom Inkspace App Helper EH in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks/Electron Helper EH.app/Contents/MacOS +1ms electron-packager Renaming Electron Helper NP to Wacom Inkspace App Helper NP in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks/Electron Helper NP.app/Contents/MacOS +0ms electron-packager Renaming Electron Helper.app to Wacom Inkspace App Helper.app in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks +1ms electron-packager Renaming Electron Helper EH.app to Wacom Inkspace App Helper EH.app in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks +0ms electron-packager Renaming Electron Helper NP.app to Wacom Inkspace App Helper NP.app in /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64/Electron.app/Contents/Frameworks +0ms electron-packager Moving /var/folders/wd/kpjczcbd28q_sbq6njl38zrw0000gn/T/electron-packager/darwin-x64/Wacom Inkspace App-darwin-x64 to ../../out/Wacom Inkspace App-darwin-x64 +2ms Wrote new app to ../../out/Wacom Inkspace App-darwin-x64

inukshuk commented 6 years ago

I just ran into a similar issue. In my case the new pruning method removes a folder in a module that contains only a package.json file. I don't know if this is the same problem as the OP but the resulting error in both cases is that a require fails, because the packaged app is missing a module. I guess this is a galactus issue?

tzvetelin-vassilev commented 6 years ago

I have created test scenario. get electron-quick-start edit render.js

const usb = require("usb");

let list = usb.getDeviceList(); console.log(list)

update package.json

"devDependencies": { "babel-core": "^6.26.3", "babel-loader": "^7.1.4", "babel-plugin-inline-react-svg": "^0.5.2", "babel-plugin-transform-remove-strict-mode": "0.0.2", "babel-preset-env": "^1.7.0", "babel-preset-react": "^6.24.1", "babel-preset-react-optimize": "^1.0.1", "babel-preset-stage-0": "^6.24.1", "clean-webpack-plugin": "^0.1.19", "concurrently": "^3.5.1", "cross-env": "^5.1.5", "css-loader": "^0.28.11", "electron": "^2.0.0", "electron-devtools-installer": "^2.2.4", "extract-text-webpack-plugin": "^3.0.2", "file-loader": "^1.1.11", "html-loader": "^0.5.5", "ignore-loader": "^0.1.2", "style-loader": "^0.21.0", "webpack": "^3.11.0" }, "dependencies": { "ag-psd": "^1.2.2", "glob": "^7.1.2", "immutable": "^3.8.2", "jpeg-js": "^0.3.4", "jwt-js": "^0.5.0", "pngjs": "^3.3.3", "prop-types": "^15.6.1", "rc-slider": "^8.6.1", "rc-tooltip": "^3.7.2", "react": "^16.3.2", "react-dom": "^16.3.2", "react-intl": "^2.4.0", "react-intl-redux": "^0.7.0", "react-list": "^0.8.7", "react-loader": "^2.4.5", "react-notification": "^6.8.2", "react-redux": "^5.0.7", "react-router": "^4.2.0", "react-router-dom": "^4.2.2", "react-router-redux": "^4.0.8", "react-scrollbar": "^0.5.4", "react-tabs": "^2.2.2", "redux": "^4.0.0", "redux-thunk": "^2.2.0", "threads": "^0.11.0", "universal-analytics": "^0.4.16", "uuid": "^3.2.1" }, "optionalDependencies": { "bluetooth-serial-port": "^2.1.6", "node-bluetooth": "^1.2.2", "gl": "^4.0.4", "leveldown": "^1.8.0", "levelup": "^1.3.9", "noble": "^1.9.1", "serialport": "^6.1.1", "usb": "^1.3.1" }

electron-packager . --asar

MarshallOfSound commented 6 years ago

I have created test scenario.

Can we lose some of the dependencies there, I'm down to fix this if it's a bug but I'm not going to go traipsing through 25+ deps to find the one causing the issue. If anyone can provide a minimal test case (quick-start with one or two extra deps) I'll take a look

tzvetelin-vassilev commented 6 years ago

OK minified package.json

"devDependencies": { "electron": "^2.0.0" }, "optionalDependencies": { "usb": "^1.3.1" }

inukshuk commented 6 years ago

@MarshallOfSound is galactus supposed to remove contents within a (direct) dependency? Because that's what is happening in my case. I have a direct dependency redux-saga; after installing the module there is a ./node_modules/redux-saga/effects/package.json (nothing else in that folder, its main pointing to a different folder); what's more, effects is not automatically loaded if you require just redux-saga (so it would be easy to miss if modules are being trimmed by default). After running electron-compile with prune enabled, redux-saga is included in the target, but the effects folder is missing. Is this something that could be caused by galactus or should I look elsewhere?

Thanks for your awesome work!

malept commented 6 years ago

@tzvetelin-vassilev usb is a native module, so:

@inukshuk I'm pretty sure galactus doesn't look for package.json files unless they're in node_modules/name_of_module/package.json. It would help if you also gave a minimal testcase.

inukshuk commented 6 years ago

Is galactus supposed to trim the contents of modules? To repeat, in my case the required module is redux-saga; this module survives the pruning just fine, but one of its sub-folders is gone. In this specific case, the folder contains a package.json but for all intents and purposes it could be an arbitrary .json file. Again, I'm not talking about a full NPM module that's gone missing, but a single folder in a module. I was under the impression that the prune option is supposed to filter out only development dependencies and does not try to further optimize the contents production dependencies.

Here is my test case:

$ mkdir tmp
$ cd tmp
$ npm init
$ npm i electron electron-packager --save-dev
$ npm i redux-saga
$ $(npm bin)/electron-packager . test --platform=linux

And then:

$ ls -l node_modules/redux-saga/effects/package.json
$ ls -l test-linux-x64/resources/app/node_modules/redux-saga/

And note that the effects folder is missing.

malept commented 6 years ago

galactus removes entire NPM modules if it is found to not be a production dependency. Of course, I just realized that it is probably a bug in Electron Packager itself. Could you file a separate bug? This is not related to the original issue.

inukshuk commented 6 years ago

Will do, thanks for taking a look at it!

malept commented 6 years ago

Given the minimal testcase of the original issue submitter, I'm fairly confident that the answer I gave in https://github.com/electron-userland/electron-packager/issues/844#issuecomment-389582609 is the reason why the package wasn't working, so I'm closing this issue.

tzvetelin-vassilev commented 6 years ago

I'm not pretty sure that asar arg fix the problem. Under Windows I don't use asar at all because squirrel problem and i have the same issue there. Also under Mac i have removed asar and package test case without any args - now i recieve 'Cannot find process-nextick-args'. node_modules is here and problem also. Of course i'm rebuilding native modules. Without this app is not able to start at all.

malept commented 6 years ago

I cannot reproduce this with your test scenario plus using electron-rebuild. I can successfully retrieve the USB device list on a packaged Linux Electron app. I can upload the code later today.

malept commented 6 years ago

Repro steps:

git clone https://github.com/malept/electron-quick-start
cd electron-quick-start
git checkout electron-packager-issue-844
npm install
npm run package
# Run packaged app, varies based on target platform/arch
tzvetelin-vassilev commented 6 years ago

Well, in fact I'm using script that eliminates need for rebuild. Here it is:

Electron's version.

export npm_config_target=$ELECTRON_VERSION

export npm_config_abi=$ELECTRON_ABI

The architecture of Electron, can be ia32 or x64.

export npm_config_arch=x64

Download headers for Electron.

export npm_config_disturl=https://atom.io/download/atom-shell

Tell node-pre-gyp that we are building for Electron.

export npm_config_runtime=electron

Tell node-pre-gyp to build module from source code.

export npm_config_build_from_source=true

Install all dependencies, and store cache to ~/.electron-gyp.

HOME=~/.electron-gyp npm install

malept commented 6 years ago

That's fine, but I still can't reproduce your problem.

In order to debug your problem further, we need a minimal testcase to reproduce your problem. Using the electron-quick-start repository as a base, could you please create a minimal Electron app that illustrates the issue you described, upload it to a GitHub repository, and post a link to it here?

tzvetelin-vassilev commented 6 years ago

This is the minimal testcase as I described.

"devDependencies": { "electron": "^2.0.1" }, "optionalDependencies": { "usb": "^1.3.1" }

--- render.js const usb = require("usb");

let list = usb.getDeviceList(); console.log(list) --- render.js

./install.sh electron-packager .

electron-packager 12.0.2 Node.js 8.9.3, Chromium 61.0.3163.100, and Electron 2.0.1

'process-nextick-args' module is not found

malept commented 6 years ago

I don't get that error, on the other hand the non-electron-rebuild script you have doesn't seem to work for me either.

Also, I don't see what calls that process-nexick-args module or what depends on it.

tzvetelin-vassilev commented 6 years ago

process-nextick-args is dependency of readable-stream. Please extend render.js with:

--- render.js const usb = require("usb");

let list = usb.getDeviceList(); console.log(list)

let device = list[0]; device.open();

console.log(device.deviceDescriptor.iSerialNumber)

device.getStringDescriptor(device.deviceDescriptor.iSerialNumber, (error, serial) => { try { device.close(); } catch(error) { console.warn(error); }

console.log(serial);

}); --- render.js

Is serial accessible? With electron-packager@11.2.0 packaged content with asar works fine.

malept commented 6 years ago

In the future, it would help if you used the "syntax highlighting" GitHub Markdown syntax, it's easier to copy/paste your code that way.

With your changes, I still can't reproduce your error on Linux.

tzvetelin-vassilev commented 6 years ago

Hi @malept ,

I have made installation on windows and linux either. In both platforms starting of packaged app fails with the same error.

Some suggestions how to proceed?

malept commented 6 years ago

If you insist on not using electron-rebuild (as I did in my example), I would suggest asking in one of the community forums, as there are more people watching those websites who could possibly help you. Rebuilding native modules falls outside of the responsibilities of Electron Packager.

tzvetelin-vassilev commented 6 years ago

I don't have troubles with electron-rebuild but I really can't make it works. So let's try again: 12.1.0 is latest version of electron-packager with supposed fix I believe. If I'm wrong please correct me. With this steps it should be works:

  1. npm install
  2. electron-rebuild usb
  3. electron-packager .

With this steps I still have the same problems. Where I'm wrong?

malept commented 6 years ago

Have you tried the repro steps in https://github.com/electron-userland/electron-packager/issues/844#issuecomment-390096316 ?

tzvetelin-vassilev commented 6 years ago

Yes, with your demo works, but it looks like workaround, not like a solution. install -> rebuild -> package = this should works

With your demo sequence is: install -> rebuild -> package (rebuild one more time)

Packaging process looses some dependencies and rebuilt recovers them. It looks creepy. Real resolution cannot impact such dependency.

malept commented 6 years ago

It's a solution, not a workaround. The packaging code in the repro steps is essentially how Electron Forge works.

tzvetelin-vassilev commented 6 years ago

I still don't understand how you neglate the fact that everything works fine before version 12.0.0 of electron-packager. Anyhow Electron Forge now works in different way - fine. "electron-packager ." executed under command line is not working. Not working for reasons that I still don't understand, but this is command line app and packager.js script proves that could work. 'afterCopy' step is nice but if this rebuilt is really needed could be command line argument or always to happens when electron-packager needs that.

malept commented 6 years ago

What likely happened is that something in npm prune did something related to your native module. As of version 12, Electron Packager no longer shells out to an external tool to prune devDependencies (nor will that change any time soon, for several reasons).

You are free to continue to use Electron Packager 11.x, if 12 is not working for you. Alternatively, if you can figure out a way to change Electron Packager so that it works with your workflow (without adding a new command line option or reintroducing npm prune), I'd be happy to review a pull request.