metacraft-labs / DendrETH

A smart contract implementation of the Ethereum light client sync protocol
GNU General Public License v3.0
45 stars 9 forks source link

Fix the emscripten package to allow the CI tests to work again #246

Open PetarKirov opened 1 year ago

PetarKirov commented 1 year ago

After upgrading Emscripten to 3.1.42, a CI job that runs the following command started failing:

nix develop -c yarn test-emcc './tests/test-nim-to-wasm.ts' 'test-nim-light-client.ts'

On closer inspection, one of the commands run by that tests is this one:

nim-wasm c --lib:"./vendor/nim/lib" -d:emcc -o:./tests/nimToWasm/arrays.wasm --d:lightClientWASM ./tests/nimToWasm/arrays.nim
Command output ``` nim-wasm c --lib:"./vendor/nim/lib" -d:emcc -o:./tests/nimToWasm/arrays.wasm --d:lightClientWASM ./tests/nimToWasm/arrays.nim Hint: used config file '/nix/store/izk7x70hprwl83588ga2jyqxd67dcsjd-x86_64-unknown-linux-gnu-nim-wrapper-1.6.14/etc/nim/nim.cfg' [Conf] Hint: used config file '/nix/store/izk7x70hprwl83588ga2jyqxd67dcsjd-x86_64-unknown-linux-gnu-nim-wrapper-1.6.14/etc/nim/config.nims' [Conf] Hint: used config file '/nix/store/608f4b7l9p128cqw4px8vihnqsly73qp-nim-cfg/nim/config.nims' [Conf] Hint: used config file './tests/nimToWasm/nim.cfg' [Conf] .................................................... Hint: emcc -shared -o ./tests/nimToWasm/arrays.wasm ~/.cache/nim/arrays_r/@z..@f..@firaqbe@favz@fyvo@fflfgrz.nim.c.o ~/.cache/nim/arrays_r/@zneenlf.nim.c.o -Oz -s ALLOW_MEMORY_GROWTH -s WASM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -ldl [Link] emcc: warning: -shared/-r used with executable output suffix. This behaviour is deprecated. Please remove -shared/-r to build an executable or avoid the executable suffix (.wasm) when building object files. [-Wemcc] Traceback (most recent call last): File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/emcc.py", line 4424, in sys.exit(main(sys.argv)) File "/nix/store/1r6n7v2wam7gkr18gxccpg7p5ywgw551-python3-3.10.12/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/emcc.py", line 4417, in main ret = run(args) File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/emcc.py", line 1320, in run js_info = get_js_sym_info() File "/nix/store/1r6n7v2wam7gkr18gxccpg7p5ywgw551-python3-3.10.12/lib/python3.10/contextlib.py", line 79, in inner return func(*args, **kwds) File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/emcc.py", line 548, in get_js_sym_info with filelock.FileLock(cache.get_path(cache.get_path('symbol_lists.lock'))): File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/tools/filelock.py", line 330, in __enter__ self.acquire() File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/tools/filelock.py", line 278, in acquire self._acquire() File "/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/tools/filelock.py", line 391, in _acquire fd = os.open(self._lock_file, open_mode) OSError: [Errno 30] Read-only file system: '/nix/store/4p9b1aap717cpjsq9539laamm20py6vr-emscripten-3.1.42/share/emscripten/cache/symbol_lists.lock' Error: execution of an external program failed: 'emcc -shared -o ./tests/nimToWasm/arrays.wasm ~/.cache/nim/arrays_r/@z..@f..@firaqbe@favz@fyvo@fflfgrz.nim.c.o ~/.cache/nim/arrays_r/@zneenlf.nim.c.o -Oz -s ALLOW_MEMORY_GROWTH -s WASM=1 -s ERROR_ON_UNDEFINED_SYMBOLS=0 -ldl' ```

The root cause is that emscripten is attempting to build some libraries to populate its cache, however this should not happen, as our package already includes a post-install step that should populate the emscripten cache with the necessary build artifacts: https://github.com/metacraft-labs/nix-blockchain-development/blob/808bdd7ba1f48d54cc80fc7a549af3cadeb8698a/packages/all-packages.nix#L131-L143