Can't create `wasm` build

[tpiros]

Describe the bug Running BUILD_TARGET=wasm32 ENABLE_WASM_SIMD=1 emconfigure ./ci.sh release yields an error

CMake Error at third_party/CMakeLists.txt:36 (message):
  Highway library (hwy) not found.  Install libhwy-dev or download it to
  third_party/highway from https://github.com/google/highway .  Highway is
  required to build JPEG XL.  You can run /root/libjxl/deps.sh to download
  this dependency.

however: apt install libhwy-dev

libhwy-dev is already the newest version (1.0.3-2).
0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded.

To Reproduce follow the wasm build instructions

Expected behavior Expected to have the wasm build ready

Screenshots

Environment

• OS: Ubuntu 23.04
• CPU type: aarch64

[mo271]

I think you might need a more up-to-date highway. Perhaps running

git submodule update --init --recursive

already fixes this problem?

[tpiros]

thanks @mo271 I will give this a go!

[tpiros]

I have managed to get past this error but I am now presented with another one.

FAILED: tools/wasm_demo/jxl_decoder.js
: && /root/emsdk/upstream/emscripten/em++ -msimd128 -pthread -fno-rtti -funwind-tables -Xclang -mrelax-all -Xclang -mconstructor-aliases -fno-omit-frame-pointer -O3 -DNDEBUG -O2 -msimd128 -pthread     -s ALLOW_MEMORY_GROWTH=1   -s DISABLE_EXCEPTION_CATCHING=1   -s MODULARIZE=1   -s USE_PTHREADS=1   -s PTHREAD_POOL_SIZE=4    -s EXPORT_NAME="JxlDecoderModule"   -s "EXPORTED_FUNCTIONS=[_free, _malloc, _jxlCreateInstance, _jxlDestroyInstance, _jxlFlush, _jxlProcessInput, _jxlDecompress, _jxlCleanup]"   -O3 -s FILESYSTEM=0 --closure 1 -mnontrapping-fptoint  -fPIE -pie tools/wasm_demo/CMakeFiles/jxl_decoder.dir/jxl_decoder.cc.o tools/wasm_demo/CMakeFiles/jxl_decoder.dir/jxl_decompressor.cc.o tools/wasm_demo/CMakeFiles/jxl_decoder.dir/no_png.cc.o -o tools/wasm_demo/jxl_decoder.js  lib/libjxl_extras-static.a  lib/libjxl.a  third_party/highway/libhwy.a  third_party/brotli/libbrotlidec.a  third_party/brotli/libbrotlicommon.a  third_party/brotli/libbrotlienc.a  lib/libjxl_threads.a  third_party/sjpeg/libsjpeg.a && :
em++: warning: -pthread + ALLOW_MEMORY_GROWTH may run non-wasm code slowly, see https://github.com/WebAssembly/design/issues/1271 [-Wpthreads-mem-growth]
Command '['/usr/bin/node', '--max_old_space_size=8192', '/root/emsdk/upstream/emscripten/node_modules/.bin/google-closure-compiler', '--platform=java', '--version']' returned non-zero exit status 254.
em++: error: closure compiler (/usr/bin/node --max_old_space_size=8192 /root/emsdk/upstream/emscripten/node_modules/.bin/google-closure-compiler --platform=java --version) did not execute properly!
ninja: build stopped: subcommand failed.
+ retcode=1
emconfigure: error: './ci.sh release' failed (returned 1)

[mo271]

Just to make sure: Are you following the steps in https://github.com/libjxl/libjxl/blob/main/doc/building_wasm.md ? Make sure you use the current main branch, since this was recently updated

It might be that the failure is caused by a incorrectly emsdk setup. What is your node version?

[tpiros]

yes, I have started everything from scratch earlier today. I installed a new Ubuntu VM and installed everything else from git.

Ubuntu 23.04 Node 20.3.0 emscripten 3.1.33

[mo271]

Is this happening for

BUILD_TARGET=wasm32 emconfigure ./ci.sh release

or

BUILD_TARGET=wasm32 ENABLE_WASM_SIMD=1 emconfigure ./ci.sh release

?

[tpiros]

A quick update, I have seen this discussion on SO, and I have removed --closure 1 from tools/wasm_demo/CMakeLists.txt.

This has solved the issue but I'm also a bit unsure about what the end result should be of a successful build?

[tpiros]

Regarding your previous question it was for BUILD_TARGET=wasm32 ENABLE_WASM_SIMD=1 emconfigure ./ci.sh release. I can try with the other command as well - one moment.

update: same result for both commands.

[tpiros]

@mo271 Let me know if removing that flag is OK and also what the final outcome should be by running that command (*). As stated earlier both emconfigure commands would return the same error (with having --closure 1 set).

*I'd love to see that added to the docs - i.e. successfully running the command would result in XYZ, and an explanation/usage instructions for those.

[mo271]

The result should be following:

$ ls build-wasm32/tools/wasm_demo/
CMakeFiles/  cmake_install.cmake  CTestTestfile.cmake  jxl_decoder_for_test.js*  jxl_decoder_for_test.wasm*  jxl_decoder_for_test.worker.js  jxl_decoder.js*  jxl_decoder.wasm*  jxl_decoder.worker.js

Then you can follow instructions here to use it: https://github.com/libjxl/libjxl/tree/main/tools/wasm_demo

I opened a pull request yesterday, adding a link to that demo in the building_wasm.md.

Regarding the --closure 1 flag: Let's check what versions exactly this is affecting, because for me and on the CI it works fine with it.

[tpiros]

Thanks @mo271 - confirmed I got those assets as well. I still need to test it but at least the files look OK.

As a side note - and I realise that this is a long shot - but why I ended up here is because initially I wanted to build SSIMULACRA v2 (http://github.com/cloudinary/ssimulacra2) and I had a few issues linking libhwy correctly with emscripten and @jonsneyers suggested that I try to build JXL for wasm first because it has the same dependencies. So I was wondering if you have ever tried to build SSIMULACRA for wasm at all?

[tpiros]

hi @mo271 - a kind follow up regarding my previous comment :)

[mo271]

Thanks for the ping @tpiros I have not tried building SSIMULACRA with wasm. Perhaps it would be useful to describe the issues you are having in more detail.

[tpiros]

I'm sorry for the awful delay here @mo271. Is there another place where we could talk (Discord or something else?)

With all transparency the issue could also stem from the fact that I have limited knowledge of the various build systems and while I was successful in compiling SSIMULACRA v1 to wasm, I can't seem to be able to do that using v2.

I have gotten as far as running the following command (after doing the git clone and just to confirm, I was also able to build SSIMULACRA itself without any issues)

emcmake cmake ../src -G Ninja

DCMAKE_TOOLCHAIN_FILE=/root/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/usr/bin/node
-- CMAKE_SYSTEM_PROCESSOR is aarch64
-- Configuring done
-- Generating done
-- Build files have been written to: /root/ssimulacra2/build

Then I tried emmake ninja ssimulacra2 but that resulted in a high nubmer of errors:

FAILED: lib/CMakeFiles/jxl_dec-obj.dir/jxl/color_management.cc.o
/usr/bin/c++ -DJPEGXL_MAJOR_VERSION=0 -DJPEGXL_MINOR_VERSION=8 -DJPEGXL_PATCH_VERSION=0 -DJXL_INTERNAL_LIBRARY_BUILD -I/root/ssimulacra2/src -I/root/ssimulacra2/src/lib/include -I/root/ssimulacra2/build/lib/include -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -fmerge-all-constants -fno-builtin-fwrite -fno-builtin-fread -Wall -Wextra -Wc++11-compat -Warray-bounds -Wformat-security -Wimplicit-fallthrough -Wno-register -Wno-unused-function -Wno-unused-parameter -Wnon-virtual-dtor -Woverloaded-virtual -Wvla -fsized-deallocation -fno-exceptions -fmath-errno -std=gnu++11 -MD -MT lib/CMakeFiles/jxl_dec-obj.dir/jxl/color_management.cc.o -MF lib/CMakeFiles/jxl_dec-obj.dir/jxl/color_management.cc.o.d -o lib/CMakeFiles/jxl_dec-obj.dir/jxl/color_management.cc.o -c /root/ssimulacra2/src/lib/jxl/color_management.cc
In file included from /root/ssimulacra2/src/lib/jxl/field_encodings.h:16,
                 from /root/ssimulacra2/src/lib/jxl/color_encoding_internal.h:24,
                 from /root/ssimulacra2/src/lib/jxl/color_management.h:18,
                 from /root/ssimulacra2/src/lib/jxl/color_management.cc:6:
/usr/local/include/hwy/ops/arm_sve-inl.h: In function ‘hwy::N_SVE::VFromD<typename D::Rebind<typename hwy::detail::Relations<typename D::T>::Unsigned> > hwy::N_SVE::IndicesFromVec(D, VI)’:
/usr/local/include/hwy/ops/arm_sve-inl.h:2364:3: error: ‘Eq’ is not a member of ‘hwy::N_SVE::detail’; did you mean ‘EqN’?
 2364 |   HWY_DASSERT(AllTrue(

(All of them are related to HWY_DASSERT)

[antermin]

@tpiros

For Discord, you can find the invitation link on https://jpegxl.info/ .

[tpiros]

Thanks @antermin. I sent you a msg over there @mo271 🙏