feat: minimal `swipl-web`

jeswr commented 2 years ago

This might be a more appropriate issue to open in https://github.com/SWI-Prolog/swipl-devel/ down the line; but I think it is fine to keep it here for now as I don't expect this will be a priority for anyone anytime soon.

It would be nice to have a much smaller build of swipl-web that does not make use of the path, fs and crypto modules; and also does not do any fetch handling internally.

This means that use cases with apps that just want to run basic queries don't need to import unecessary code into their apps.

I'm analyse tree-shaking downstream can offer for this particular file; which could well make this a moot point to solve anyway.

rla commented 2 years ago

I believe this depends on https://github.com/emscripten-core/emscripten/issues/5828. It would likely make cross-environment usage also easier. _"Want emulated filesystem? Just import @something/emulatedfilesystem and mount to the Module".

Regarding loadable code size, it's dominated by .wasm and .data (Prolog), so the gain might not be that good:

swipl-web.data 5.5 MiB
swipl-web.wasm 1.8MiB
swipl.js 174.4 KiB

There is also WASI-SDK which has own libc, uses similar compiler setup with clang as Emscripten but does not build any sort of JS at all. libc (also posix?) emulation is done inside wasm binary, not outside: https://github.com/WebAssembly/wasi-sdk. Might be worth to investigate.

rla commented 2 years ago

Code size regarding C/wasm is also discussed in https://swi-prolog.discourse.group/t/wiki-discussion-swi-prolog-in-the-browser-using-wasm/5651/109

We definitely want to distribute the full build with as much included as possible but also the/a minimal build? For the minimal build:

Include no packages?
Include no boot/library, is this useful for something?

The NPM package can contain things side-by-side in dist subdirectories. It would not be too difficult to make the Docker-based build produce binaries in different weight classes. Anyone needing something really minimal could always build their own too.

rla commented 2 years ago

I also noticed that swipl-web.js contains references to modules path, fs and crypto while creating the Webpack example. But all these were only used when the running environment is node. It is interesting that this is inside supposedly a browser code. Replacing these modules with empty ones bundle-time will work:

https://github.com/rla/npm-swipl-wasm/blob/1057c488e777989ee0b1c9dc34878c82da444e2e/examples/webpack/webpack.config.js#L12

jeswr commented 1 year ago

It might also be worthwhile investigating the use of the closure compiler to reduce the size of the emitted JS. I did quickly try it but it looks like some fixes are required on pre-js and post-js first.

FAILED: src/swipl-web.js 
: && /emsdk/upstream/emscripten/emcc -O3 -DNDEBUG -O3 -s WASM=1 -s MODULARIZE=1 -s EXPORT_NAME=SWIPL -s NO_EXIT_RUNTIME=0 -s WASM_BIGINT=1 -s ALLOW_MEMORY_GROWTH=1 -s EXPORTED_FUNCTIONS=@/swipl-devel/src/wasm/exports.json -s EXPORTED_RUNTIME_METHODS=@/swipl-devel/src/wasm/runtime_exports.json -s SINGLE_FILE -s ELIMINATE_DUPLICATE_FUNCTIONS=1 --pre-js /swipl-devel/src/wasm/pre.js --post-js /swipl-devel/src/wasm/prolog.js --closure 1 src/CMakeFiles/swipl-web.dir/pl-main.c.o -o src/swipl-web.js  src/libswipl.a  /gmp/lib/libgmp.a  /zlib-1.2.12/libz.a  -lm  -lrt  /zlib-1.2.12/libz.a  -lm  -lrt  packages/clib/memfile.a  packages/clib/files.a  packages/clib/uri.a  packages/clib/readutil.a  packages/clib/prolog_stream.a  packages/clib/md54pl.a  packages/clib/crypt.a  packages/clib/hashstream.a  packages/clib/sha4pl.a  packages/http/json.a  packages/http/http_stream.a  packages/semweb/turtle.a  packages/semweb/ntriples.a  packages/sgml/sgml2pl.a  packages/nlp/double_metaphone.a  packages/nlp/porter_stem.a  packages/nlp/isub.a  packages/nlp/snowball.a  packages/nlp/libstemmer_c/liblibstemmer.a  packages/zlib/zlib4pl.a && :
building:ERROR: Closure compiler run failed:

building:ERROR: /tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js:5605:4: WARNING - [JSC_MISPLACED_ANNOTATION] Misplaced function annotation. This JSDoc is not attached to a function node. Are you missing parentheses?
  5605|  */ class Prolog {
            ^^^^^^^^^^^^^^
  5606|  constructor(module, args) {
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
  6530|  }
        ^^
  6531| }
        ^

/tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js:6071:20: WARNING - [JSC_TYPE_PARSE_ERROR] Bad type annotation. expected closing } See https://github.com/google/closure-compiler/wiki/Annotating-JavaScript-for-the-Closure-Compiler for more information.
  6071|  * @return {IOSTREAM*} as a number (pointer)
                            ^

/tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js:6548:4: WARNING - [JSC_MISPLACED_ANNOTATION] Misplaced function annotation. This JSDoc is not attached to a function node. Are you missing parentheses?
  6548|  */ class Query {
            ^^^^^^^^^^^^^
  6549|  constructor(prolog, module, flags, pred, argv, map, fid) {
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
  6716|  }
        ^^
  6717| }
        ^

/tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js:5919:33: ERROR - [JSC_REASSIGNED_CONSTANT] Constant reassigned: size
  5919|     if (!size instanceof Number) size = -1;
                                         ^^^^

/tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js:6096:85: ERROR - [JSC_REFERENCE_BEFORE_DECLARE_ERROR] Illegal variable reference before declaration: flags
  6096|   const flags = opts.flags == undefined ? this.PL_WRT_QUOTED | this.PL_WRT_NEWLINE : flags;
                                                                                             ^^^^^

2 error(s), 3 warning(s)

emcc: error: closure compiler failed (rc: 2): /emsdk/node/14.18.2_64bit/bin/node --max_old_space_size=8192 /emsdk/upstream/emscripten/node_modules/.bin/google-closure-compiler --compilation_level ADVANCED_OPTIMIZATIONS --language_in ECMASCRIPT_2020 --language_out NO_TRANSPILE --emit_use_strict=false --externs /emsdk/upstream/emscripten/src/closure-externs/closure-externs.js --externs /emsdk/upstream/emscripten/src/closure-externs/node-externs.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/dgram.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/querystring.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/process.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/vm.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/child_process.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/tty.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/zlib.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/string_decoder.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/stream.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/repl.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/os.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/readline.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/buffer.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/tls.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/domain.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/util.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/https.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/dns.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/punycode.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/assert.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/net.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/cluster.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/events.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/http.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/path.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/fs.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/url.js --externs /emsdk/upstream/emscripten/third_party/closure-compiler/node-externs/core.js --js /tmp/emscripten_temp_aodu0gm6/swipl-web.js.pp.js.jso.js.jso.js.jso.js.jso.js --js_output_file tmpctaattkc.cc.js the error message may be clearer with -g1 and EMCC_DEBUG=2 set
ninja: build stopped: subcommand failed.
The command '/bin/sh -c /build-swipl.sh' returned a non-zero code: 1
ERROR: "build:wasm-docker:build" exited with 1.
ERROR: "build:wasm-docker" exited with 1.

rla commented 1 year ago

Seems like code is already minified. I'm not sure how much closure compiler could reduce it further.

It would still be nice to fix to errors. Seems like closure compiler wants to interpret jsdoc very strictly. Maybe the check can be turned off or those jsdoc nodes moved to constructors or otherwise fixed.

Error in if (!size instanceof Number) size = -1 seems to be fixed already.

Variable : flags in ternary-if should probably be opts.flags:

https://github.com/SWI-Prolog/swipl-devel/blob/d65e0686bc68da7525355c6b583e5784620258d0/src/wasm/prolog.js#L805

SWI-Prolog / npm-swipl-wasm

feat: minimal `swipl-web` #4