search

electron-userland / electron-builder

A complete solution to package and build a ready for distribution Electron app with “auto update” support out of the box
https://www.electron.build
MIT License
13.68k stars 1.74k forks source link

Prebuild binaries not being downloaded #1415

Closed neurosnap closed 7 years ago

neurosnap commented 7 years ago

It doesn't seem like electron-builder is attempting to use prebuild-install at all.

DEBUG=electron-builder ./node_modules/.bin/build -m --x64
  electron-builder Two package.json structure is used (dev: /Users/erock/notion/desktop/package.json, app: /Users/erock/notion/desktop/app/package.json) +0ms
  electron-builder Effective config: {
  electron-builder   "npmSkipBuildFromSource": true,
  electron-builder   "appId": "ai.notion.desktop",
  electron-builder   "asarUnpack": [
  electron-builder     "**/*.node"
  electron-builder   ],
  electron-builder   "publish": [
  electron-builder     {
  electron-builder       "provider": "s3",
  electron-builder       "bucket": "notion-desktop"
  electron-builder     },
  electron-builder     {
  electron-builder       "provider": "github"
  electron-builder     }
  electron-builder   ],
  electron-builder   "dmg": {
  electron-builder     "contents": [
  electron-builder       {
  electron-builder         "x": 410,
  electron-builder         "y": 150,
  electron-builder         "type": "link",
  electron-builder         "path": "/Applications"
  electron-builder       },
  electron-builder       {
  electron-builder         "x": 130,
  electron-builder         "y": 150,
  electron-builder         "type": "file"
  electron-builder       }
  electron-builder     ]
  electron-builder   },
  electron-builder   "mac": {
  electron-builder     "category": "public.app-category.productivity"
  electron-builder   },
  electron-builder   "linux": {
  electron-builder     "target": [
  electron-builder       "deb",
  electron-builder       "AppImage"
  electron-builder     ]
  electron-builder   }
  electron-builder } +3ms
Rebuilding native production dependencies for darwin:x64
  electron-builder Spawning npm rebuild notion-spellchecker +368ms

> notion-spellchecker@3.3.2 install /Users/erock/notion/desktop/app/node_modules/notion-spellchecker
> prebuild-install || node-gyp rebuild

  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/affentry.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/affixmgr.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/csutil.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/dictmgr.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/filemgr.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/hashmgr.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/hunspell.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/hunzip.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/phonet.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/replist.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/hunspell/suggestmgr.o
  CXX(target) Release/obj.target/hunspell/vendor/hunspell/src/parsers/textparser.o
  LIBTOOL-STATIC Release/hunspell.a
  CXX(target) Release/obj.target/spellchecker/src/main.o
  CXX(target) Release/obj.target/spellchecker/src/spellchecker_hunspell.o
  CXX(target) Release/obj.target/spellchecker/src/spellchecker_mac.o
  CXX(target) Release/obj.target/spellchecker/src/transcoder_posix.o
  SOLINK_MODULE(target) Release/spellchecker.node
notion-spellchecker@3.3.2 /Users/erock/notion/desktop/app/node_modules/notion-spellchecker
develar commented 7 years ago

Well, prebuild-install || node-gyp rebuild means that no prebuild binary for your electron versions and arch.

neurosnap commented 7 years ago

Hmm, doing a npm rebuild runs the npm install command which is what that is, doesn't that have to run for prebuild to do its magic?

Also, how can I confirm if the prebuilt binary exists? I'm not really seeing any relevant logging in this output.

neurosnap commented 7 years ago

Here is the relevant release with binaries: https://github.com/notion/node-spellchecker/releases/tag/v3.3.3

develar commented 7 years ago

doing a npm rebuild runs the npm install command which is what that is, doesn't that have to run for prebuild to do its magic?

You cannot do npm rebuild, only tool can do — arch, target electron version, headers must be set.

how can I confirm if the prebuilt binary exists

Please ask prebuild developers.

Here is the relevant release with binaries

Hmm.... ABI version is 53, should be ok. I will check. Could you please share your project?

neurosnap commented 7 years ago

You cannot do npm rebuild, only tool can do — arch, target electron version, headers must be set.

Right, I was saying that when I manually ran npm rebuild it detected the binaries just fine.

When I removed this piece of code the build was able to continue without compiling but then the build failed entirely.

GNU xorriso 1.4.4 : RockRidge filesystem manipulator, libburnia project.

xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
Drive current: -dev '/Users/****/jenkins/workspace/Desktop-Build/dist/Notion-0.20.2-x86_64.AppImage'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data,  731g free
New volume id: 'AppImage'
xorriso : UPDATE : 106 files added in 1 seconds
Added to ISO image: directory '/usr/bin'='/Users/****/jenkins/workspace/Desktop-Build/dist/linux-unpacked'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/AppRun'='/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/templates/linux/AppRun.sh'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/notion.desktop'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-5-Notion.desktop'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/16x16/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_16x16.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/24x24/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_24x24.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/32x32/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_32x32.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/48x48/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_48x48.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/64x64/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_64x64.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/96x96/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_96x96.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/128x128/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_128x128.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/256x256/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_256x256.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/512x512/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_512x512.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/.DirIcon'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_512x512.png'
xorriso : UPDATE : 6 files added in 1 seconds
Added to ISO image: directory '/usr/lib'='/Users/****/Library/Caches/electron-builder/AppImage-packages/10.03.17'
xorriso : UPDATE : 8 file filters processed in 1 seconds
xorriso : UPDATE : 9 file filters processed in 5 seconds
xorriso : UPDATE : 10 file filters processed in 7 seconds
xorriso : UPDATE : 11 file filters processed in 16 seconds
xorriso : UPDATE : 67 file filters processed in 32 seconds
xorriso : UPDATE : 68 file filters processed in 36 seconds
libisofs: FAILURE : Filter input differs from previous run
xorriso : UPDATE : 108 file filters processed in 37 seconds
xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'

GNU xorriso 1.4.4 : RockRidge filesystem manipulator, libburnia project.

xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
Drive current: -dev '/Users/****/jenkins/workspace/Desktop-Build/dist/Notion-0.20.2-x86_64.AppImage'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data,  731g free
New volume id: 'AppImage'
xorriso : UPDATE : 106 files added in 1 seconds
Added to ISO image: directory '/usr/bin'='/Users/****/jenkins/workspace/Desktop-Build/dist/linux-unpacked'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/AppRun'='/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/templates/linux/AppRun.sh'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/notion.desktop'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-5-Notion.desktop'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/16x16/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_16x16.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/24x24/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_24x24.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/32x32/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_32x32.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/48x48/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_48x48.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/64x64/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_64x64.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/96x96/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_96x96.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/128x128/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_128x128.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/256x256/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_256x256.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/usr/share/icons/default/512x512/apps/notion.png'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_512x512.png'
xorriso : UPDATE : 1 files added in 1 seconds
Added to ISO image: file '/.DirIcon'='/var/folders/yp/mhyppj_15z15cl2bf1_cn7jm0000gn/T/electron-builder-x3oJa3/0-4-linux.iconset/icon_512x512.png'
xorriso : UPDATE : 6 files added in 1 seconds
Added to ISO image: directory '/usr/lib'='/Users/****/Library/Caches/electron-builder/AppImage-packages/10.03.17'
xorriso : UPDATE : 8 file filters processed in 1 seconds
xorriso : UPDATE : 9 file filters processed in 5 seconds
xorriso : UPDATE : 10 file filters processed in 7 seconds
xorriso : UPDATE : 11 file filters processed in 16 seconds
xorriso : UPDATE : 67 file filters processed in 32 seconds
xorriso : UPDATE : 68 file filters processed in 36 seconds
libisofs: FAILURE : Filter input differs from previous run
xorriso : UPDATE : 108 file filters processed in 37 seconds
xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'

    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/node_modules/electron-builder-util/src/util.ts:75:16
    at ChildProcess.exithandler (child_process.js:212:5)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:194:7)
    at maybeClose (internal/child_process.js:899:16)
    at Socket.<anonymous> (internal/child_process.js:342:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:191:7)
    at Pipe._handle.close [as _onclose] (net.js:513:12)
From previous event:
    at exec (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/node_modules/electron-builder-util/src/util.ts:53:3)
    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/targets/appImage.ts:79:5
    at Generator.next (<anonymous>)
From previous event:
    at AppImageTarget.build (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/targets/appImage.js:144:11)
    at postAsyncTasks.push.default.map.it (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/platformPackager.ts:106:85)
From previous event:
    at LinuxPackager.packageInDistributableFormat (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/platformPackager.ts:106:41)
    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/platformPackager.ts:102:10
From previous event:
    at LinuxPackager.pack (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/platformPackager.js:169:11)
    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/packager.ts:257:24
From previous event:
    at Packager.doBuild (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/packager.js:379:11)
    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/packager.ts:170:52
    at Generator.next (<anonymous>)
    at runCallback (timers.js:666:20)
    at tryOnImmediate (timers.js:639:5)
    at processImmediate [as _immediateCallback] (timers.js:611:5)
From previous event:
    at Packager.build (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/packager.js:274:11)
    at /Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/src/builder.ts:198:33
    at Generator.next (<anonymous>)
From previous event:
    at build (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/builder.js:70:21)
    at Object.<anonymous> (/Users/****/jenkins/workspace/Desktop-Build/node_modules/electron-builder/out/cli/build-cli.js:71:41)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:427:7)
    at startup (bootstrap_node.js:148:9)
    at bootstrap_node.js:542:3
Succeeded

Unfortunately I cannot, it's a private repo. We are going to end up building the apps on each target OS.

lgeiger commented 7 years ago

@develar We run into the same issue on nteract. Here is a example build log.

The prebuilts are available for every Electron version and can manually rebuild correctly without building from source.

It looks like somehow the npmSkipBuildFromSource=true flag isn't respected.

lgeiger commented 7 years ago

@develar Do you have a idea where I could look into to track down the exact issue?

develar commented 7 years ago

@lgeiger I will investigate soon. I have no idea (I will just debug step by step :) and will investigate how to get full debug log from prebuild).

lgeiger commented 7 years ago

That's awesome! Thanks 👍

lgeiger commented 7 years ago

I just tried again with electron-builder v16.8.0 and it still fails.

electron-builder tries to build against target=7.8.0 (my local node version) and runtime=electron. Therefore prebuild-install can't find the relevant version, since Electron version 7.8.0 doesn't exist.

I think config.muonVersion == null ? null : getElectronVersion(config, projectDir) is causing the issue.

I don't know what config.muonVersion does but I guess this line should say something similar to

config.muonVersion || getElectronVersion(config, projectDir)
develar commented 7 years ago

@lgeiger Thanks, I have fixed this regression fixed in 16.8.1.

develar commented 7 years ago

@neurosnap node-spellchecker works for me now (16.8.1), could you please verify?

lgeiger commented 7 years ago

@develar 16.8.1 definitely fixed the regression described above, but doesn't fix the original issue for me: https://travis-ci.org/lgeiger/nteract/jobs/220309476#L749

lgeiger commented 7 years ago

Using #1450 I was able to debug this issue. I made a PR to prebuild-install which should fix this issue.

@neurosnap As a workaround you can add mkdir ~/.electron-gyp to the before_install script in travis.yml until the issue is fixed.

lgeiger commented 7 years ago

This issue was fixed in prebuild-install 2.1.2: https://github.com/mafintosh/prebuild-install/pull/33