can't build v0.804.0 with npm@6.9.0 due to "cb() never called!"

[muelli]

I'm trying to build the v0.201.0 version but npm install fails:

Running: npm install --offline --cache=/run/build/delta/npm-cache/ --verbose --dc-system-lib=true
FB: Running: flatpak build --die-with-parent --env=FLATPAK_BUILDER_BUILDDIR=/run/build/delta --nofilesystem=host --filesystem=/export/home/muelli/vcs/chat.delta.desktop/.flatpak-builder/build/delta-6 --bind-mount=/run/build/delta=/export/home/muelli/vcs/chat.delta.desktop/.flatpak-builder/build/delta-6 --build-dir=/run/build/delta --bind-mount=/run/ccache=/export/home/muelli/vcs/chat.delta.desktop/.flatpak-builder/ccache --env=SOURCE_DATE_EPOCH=1571054044 --env=NPM_CONFIG_LOGLEVEL=info --env=ELECTRON_CACHE=/run/build/delta/npm-cache --env=npm_config_devdir=/app --env=npm_config_nodedir=/app/ --env=electron_config_cache=/run/build/delta/npm-cache --env=CCACHE_DIR=/run/ccache --env=PATH=/run/ccache/bin:/app/bin:/usr/bin --env=LD_LIBRARY_PATH=/app/lib --env=PKG_CONFIG_PATH=/app/lib/pkgconfig:/app/share/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig --env=FLATPAK_BUILDER_N_JOBS=32 /export/home/muelli/vcs/chat.delta.desktop/.flatpak-builder/rofiles/rofiles-mZ3ggh /bin/sh -c 'npm install --offline --cache=/run/build/delta/npm-cache/ --verbose --dc-system-lib=true'
npm info it worked if it ends with ok
npm verb cli [ '/app/bin/node',
npm verb cli   '/app/bin/npm',
npm verb cli   'install',
npm verb cli   '--offline',
npm verb cli   '--cache=/run/build/delta/npm-cache/',
npm verb cli   '--verbose',
npm verb cli   '--dc-system-lib=true' ]
npm info using npm@6.9.0
npm info using node@v10.16.3
npm verb npm-session 86c250c00da16da5
npm info lifecycle deltachat-desktop@0.201.0~preinstall: deltachat-desktop@0.201.0
npm timing stage:loadCurrentTree Completed in 16ms
npm timing stage:loadIdealTree:cloneCurrentTree Completed in 0ms
npm timing stage:loadIdealTree:loadShrinkwrap Completed in 963ms
npm timing stage:loadIdealTree:loadAllDepsIntoIdealTree Completed in 1272ms
npm timing stage:loadIdealTree Completed in 2769ms
npm timing stage:generateActionsToTake Completed in 502ms
npm verb correctMkdir /run/build/delta/npm-cache/_locks correctMkdir not in flight; initializing
npm verb makeDirectory /run/build/delta/npm-cache/_locks creation not in flight; initializing
npm verb lock using /run/build/delta/npm-cache/_locks/staging-dbc36da217aa0b65.lock for /run/build/delta/node_modules/.staging
npm timing npm Completed in 11762ms
npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>

npm ERR! A complete log of this run can be found in:
npm ERR!     /run/build/delta/npm-cache/_logs/2019-10-14T12_02_13_217Z-debug.log
Error: module delta: Child process exited with code 1

The npm log is here. The flatpak manifest for reproducing the problem is here: https://github.com/flathub/chat.delta.desktop/blob/fb463decd239dd3a0d92d9e49bb063262b761fd6/chat.delta.desktop.yml. The build log is here: https://flathub.org/builds/#/builders/19/builds/1175

[muelli]

@flub maybe you can have a look here and reproduce, at least?

[flub]

This sounds eerily familiar... I've been stuck on this for ages before once, can't remember how it got solved though. I'll try and do some faltpak work soon, seems like the releases are coming closer as well.

[muelli]

cool. Is this a flatpak thing, though? It looks like a general npm thingy to me.

[muelli]

the same problem with the newer 0.804.0 release.

[Jikstra]

Seems like a npm issue, not sure what to do about it besides using another npm version. nvm is quite useful in this case to easily test different node/npm versions. Besides this, i'm currently not seeing any real world benefit of fixing all those various things to make flatpaks work, as the AppImages which we ship since many releases work without a problem. In my opinion sandboxing and having every dependency again for flatpak is a complete overkill for an electron app. Besides this the flatpak build system seems to be quite complicate and picky, also the core desktop team needs to fix those issues without them knowing a thing about flatpak, and at least I for myself am not really interested in diving into this topic. Please tell me why you think flatpak is superior and definetly needed, otherwise i would like to drop any efforts to make this happen. Thanks a lot for all your efforts of course in this topic!

[muelli]

Turns out, that this is indeed an npm issue. I would love to link to a bug report, but it seems the community has yet not managed to have a bug tracker. I have found someone reporting a similar behaviour https://github.com/RocketChat/Rocket.Chat/issues/15408 which indicates a bug in npm. The "fix" is to amend the package-lock.json file as indicated in that issue.

I am not going to fight over what the superior app distribution format is and refer to the docs. Let me just refute your assessment of "the various things to make flatpaks work". Requiring to know what software components an app is made of and thus being able to spell the actual addresses and hashes of what goes in an app is fashionable these days. Especially for security related software. Bonus points if the builds are reproducible. Having high quality appdata makes the experience on GNU/Linux systems much better, because it offers integration points for various software catalogues. Thus, there is no flatpak-specific work involved. If you ever attempt to get the app into any reputable distribution, say Debian, you will face similar, if not the same, problems.

[Jikstra]

@muelli i'm not against fixing those things, i'm just noticing that we have quite some stalled issues for flatpak, which didn't get fixed in months. Keeping the appdata file up2date makes sense, but currently it's one more step of making a release, and doing releases currently already takes multiple hours waiting for the ci to finish building stuff, building installs on our own because we didn't find a way to do it automated yet (having signing keys on a strangers ci is not really good for security), curating changelogs and being careful with git tags. Also taking care of the appdata file, which i never saw anywhere used, but being a requirement asked for flatpaks is just some extra work I currently don't want to do, and also don't want to put effort in to write a script to make this easier. This doesn't mean that i'm against having this, it's just that i'm currently not focusing on this. So whoever thinks this is important and needed, i'm very happy to review and merge prs doing this.