Open shamrin opened 1 year ago
That looks very odd indeed. I would start be trying to figure out what is different about these working and failing builds.
Does the resulting Makefile
looks different? How?
Does the working build also #define malloc rpl_malloc
? Where does it get its definition of rpl_malloc from?
@sbc100 very odd, yes.
Makefile
is the same:$ diff -u liquid-dsp-good/makefile liquid-dsp-bad/makefile
# no output
#define malloc rpl_malloc
. Working build:$ grep rpl_malloc config.h
config.h:/* Define to rpl_malloc if the replacement function should be used. */
Broken build:
$ grep rpl_malloc config.h
config.h:/* Define to rpl_malloc if the replacement function should be used. */
config.h:#define malloc rpl_malloc
You can see the full difference between the builds at the end of this comment. But the most interesting thing is in config.log
:
$ diff -u liquid-dsp-good/config.log liquid-dsp-bad/config.log
@@ -103,8 +103,21 @@
configure:3546: /Users/shamrin/src/emsdk/upstream/emscripten/emcc -o conftest conftest.c >&5
configure:3550: $? = 0
configure:3557: ./conftest
-configure:3561: $? = 0
-configure:3576: result: no
+internal/process/esm_loader.js:74
+ internalBinding('errors').triggerUncaughtException(
+ ^
+
+TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for /Users/shamrin/src/emscripten-bug/liquid-dsp-bad/conftest
+ at new NodeError (internal/errors.js:322:7)
+ at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:71:15)
+ at Loader.getFormat (internal/modules/esm/loader.js:105:42)
+ at Loader.getModuleJob (internal/modules/esm/loader.js:243:31)
+ at async Loader.import (internal/modules/esm/loader.js:177:17)
+ at async Object.loadESM (internal/process/esm_loader.js:68:5) {
+ code: 'ERR_UNKNOWN_FILE_EXTENSION'
+}
+configure:3561: $? = 1
+configure:3576: result: yes
configure:3581: checking for suffix of object files
configure:3604: /Users/shamrin/src/emsdk/upstream/emscripten/emcc -c conftest.c >&5
configure:3608: $? = 0
Interestingly, the exception is the same as in #13551 🤔
I think I figured out what is going on .. its node actually that is being effected by the package.json files, and we are using node to run the emscripten executable "conftest" binaries.
@sbc100 Hmm, interesting.
Apparently, node
does not have a straightforward way to force commonjs
. It's either extension (.mjs
vs .cjs
) or type: "commonjs"
in package.json
: https://nodejs.org/api/packages.html
However, I found a promising comment in the long GitHub issue: https://github.com/nodejs/node/issues/37848#issuecomment-803878498
Can we launch node
with the following invocation? How can I try it out? I'm not familiar with the way emscripten calls node
.
node --experimental-loader 'data:text/javascript,export async function getFormat() { return { format: "commonjs" } }' ./index.js
@sbc100 Hmm, interesting.
Apparently,
node
does not have a straightforward way to forcecommonjs
. It's either extension (.mjs
vs.cjs
) ortype: "commonjs"
inpackage.json
: https://nodejs.org/api/packages.htmlHowever, I found a promising comment in the long GitHub issue: nodejs/node#37848 (comment)
Can we launch
node
with the following invocation? How can I try it out? I'm not familiar with the way emscripten callsnode
.node --experimental-loader 'data:text/javascript,export async function getFormat() { return { format: "commonjs" } }' ./index.js
Hmm.. that is not very pretty.
Look for make_js_executable
in emcc.py
@sbc100 data:text/javascript
trick didn't work, but providing an .mjs
file did. I've submitted PR #17451 with the fix.
I think I have an idea for how to deal with this.
Create conftest
as a shell script that then runs node conftest.cjs
and generate the actual code in conftest.cjs
Adding type: "commonjs"
to Emscripten's package.json isn't enough?
I think the problem is that the generated files don't live under the emscripten directory.. they live outside in user's own projects.
Oh yeah.
Your idea of make a shellscrip run a .cjs sounds solid in that case. If conftest won't run a file with an extension.
Please see my https://github.com/emscripten-core/emscripten/pull/17451#issuecomment-1902274071 for a possible fix.
Emscripten build of a C library (liquid-dsp) mysteriously fails when the parent directory has
package.json
containing{"type":"module"}
.There is a similar bug #13551, from February 2021, with different library and different symptoms. But with exactly the same core reason of
{"type":"module"}
changing Emscripten behaviour in a subdirectory.I spent two hours debugging this problem. It breaks the fundamental assumption that nothing outside of the project directory should matter during the build.
Any ideas about why it could be happening? I would love to help fix it in Emscripten.
Failing command line in full:
Build mysteriously fails with:
Version of emscripten/emsdk: