Esbuild-wasm fails on angular optimization on s390x

[AlvinChacko]

I cant seem to find the correct source of this problem here. Im trying to build an angular application on z and everytime the optimization is enabled I get this error. The error goes away when optimization is disabled which I dont want to do. Any clue on why this error shows up?

ERROR in Optimization error [main.7d29a25361c73fea.js]: Error: open /tmp/esbuild-294b92d156a40204c6b71e48505033c5cd3db58fef43db7bbe12abba1340ecee: Invalid argument
    at ui/node_modules/esbuild-wasm/lib/main.js:806:29
    at responseCallbacks.<computed> (ui/node_modules/esbuild-wasm/lib/main.js:687:9)
    at handleIncomingPacket (ui/node_modules/esbuild-wasm/lib/main.js:740:9)
    at Socket.readFromStdout (ui/node_modules/esbuild-wasm/lib/main.js:663:7)
    at Socket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:335:12)
    at readableAddChunk (node:internal/streams/readable:308:9)
    at Readable.push (node:internal/streams/readable:245:10)
    at Pipe.onStreamRead (node:internal/stream_base_commons:192:23)
Error: Optimization error [main.7d29a25361c73fea.js]: Error: open /tmp/esbuild-294b92d156a40204c6b71e48505033c5cd3db58fef43db7bbe12abba1340ecee: Invalid argument
    at ui/node_modules/esbuild-wasm/lib/main.js:806:29
    at responseCallbacks.<computed> (ui/node_modules/esbuild-wasm/lib/main.js:687:9)
    at handleIncomingPacket (ui/node_modules/esbuild-wasm/lib/main.js:740:9)
    at Socket.readFromStdout (ui/node_modules/esbuild-wasm/lib/main.js:663:7)
    at Socket.emit (node:events:517:28)
    at addChunk (node:internal/streams/readable:335:12)
    at readableAddChunk (node:internal/streams/readable:308:9)
    at Readable.push (node:internal/streams/readable:245:10)
    at Pipe.onStreamRead (node:internal/stream_base_commons:192:23)
    at addError (ui/node_modules/@angular-devkit/build-angular/src/utils/webpack-diagnostics.js:16:29)
    at ui/node_modules/@angular-devkit/build-angular/src/webpack/plugins/javascript-optimizer-plugin.js:152:64
    at async Promise.all (index 5)
    at async ui/node_modules/@angular-devkit/build-angular/src/webpack/plugins/javascript-optimizer-plugin.js:155:21

> ng version

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/

Angular CLI: 15.2.10
Node: 18.18.2
Package Manager: npm 9.8.1
OS: os390 s390x

Angular: 15.2.10
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1502.10
@angular-devkit/build-angular   15.2.10
@angular-devkit/core            15.2.10
@angular-devkit/schematics      15.2.10
@angular/cdk                    15.2.9
@angular/material               15.2.9
@schematics/angular             15.2.10
rxjs                            7.8.1
typescript                      4.9.5
webpack                         5.82.0

npm ls esbuild-wasm
`-- @angular-devkit/build-angular@15.2.10
  `-- esbuild-wasm@0.17.8

[evanw]

It looks like the error is coming from the use of esbuild's transform API. Specifically when transforming a string that's larger than a certain size, it's faster to write it to the file system and then read it back in instead of using node's slow stream API for inter-process communication. This is used to transfer content from node (JavaScript) to esbuild (Go) and back. It's explained in this comment:

https://github.com/evanw/esbuild/blob/a652e730ff07b9081470ef6965f3d54daa7b2aab/lib/shared/common.ts#L704-L718

If I had to guess, I'd say it looks like node/JS is writing a temporary file and then passing that temporary file name to esbuild/Go but then esbuild/Go is unable to read it so it returns the error starting with open to node/JS which then throws it.

Fixing this would require debugging why the file is inaccessible to the esbuild child process on your system. Is the file even created in the first place? Is something deleting it immediately after it's created? Does s390x have an unusual approach to permissions where a child process can't read files created by the parent process (I know nothing about s390x)? I won't be able to do that level of debugging because I don't have access to that operating system.

[AlvinChacko]

Ahh that makes sense. So the file is getting created because right after the build I am able to view the contents of the file with no issues. The permission can be a problem because maybe my home directory dont have access to the /tmp location. When you say temp file something like this right: Optimization error [654.c8a4b2a0f2e0a858.js]: Error: open /tmp/esbuild-1de042f0cf6eae00b5d40e8a995e06cf2313407b8493b523bb0a2717f1e3a1de: Invalid argument

/tmp/esbuild-1de042f0cf6eae00b5d40e8a995e06cf2313407b8493b523bb0a2717f1e3a1de

If Im able to view it and it gets created in tmp directory I thought it would be past those permission checks. Curious on why esbuild is not able to read it. Is there any flag that I can set to see little bit more descriptive error or what in the reading it fails.

Also to mention this process does not happen when optimization is disabled? Because the npm build will finish successfully when optimization is set to false.

[evanw]

Sorry, it looks like the actual problem might be some issue that the Go standard library is having with your operating system. Once the Go code gets the file path it calls Go's ioutil.ReadFile function and my best guess is that this is what's returning the error open ...: Invalid argument. I assume that is happening because the operating system is passing EINVAL to the Go standard library for something it tries to do inside ioutil.ReadFile. However, I could be wrong here because it's your system, not mine, and I'm just guessing without actually debugging. EINVAL might not mean that the file is inaccessible, since you'd expect that to be ENOENT or EACCES or something. I don't know what it would mean.

The source code for ReadFile is here. It calls Open followed by Stat (to get the size) then some number of Read calls followed by Close. One of the underlying syscalls there might be returning EINVAL. I haven't ever had to debug something like this, but I imagine using some utility that traces syscalls (strace?) would be insightful (to figure out where the EINVAL is coming from and what the invalid arguments are).

It's also possible to use a debug build of esbuild with extra logging code added (e.g. fmt.Fprintf(os.Stderr, "got here\n")) to confirm that the error is indeed coming from that particular call to ioutil.ReadFile. To do that, you could check out this repo at the tag corresponding to the version of esbuild you use, add the debug logging, compile esbuild, and then run your program with the ESBUILD_BINARY_PATH environment variable set to the compiled binary executable. You'd need to print to stderr as esbuild uses stdout as an IPC channel with node.

Also to mention this process does not happen when optimization is disabled? Because the npm build will finish successfully when optimization is set to false.

I don't know what "optimization" means here. It could be some Angular-specific thing (I don't know anything about Angular). If it basically means "run esbuild or not" then it could be that disabling optimization in Angular ends up not calling esbuild.

[AlvinChacko]

Okay so I downloaded the GO compiler for my OS and built esBuild out of that. Now I see my print statements and I dont run into the error at all. Everything works fine. I couldnt verify if the ReadFile is failing here but makes sense that when the right compiler was used, it works just fine. So the theory of where it was failing fits in the picture.

I also tried setting the GO compiler in the environment before running a npm ci but that didnt help. How can I make sure that when installing esbuild using npm it compiles using the GO compiler that is specified in the environment?

[evanw]

Go is a cross-compiler. It's not supposed to matter which operating system you compile your code on (as long as you aren't using cgo, which esbuild explicitly disables). For example, every time esbuild is released, the published binaries are automatically verified to be bitwise-identical between macOS (where I publish from) and Linux (where they are verified).

It could matter which version of the Go compiler that you use, however. The published binary executables for esbuild are currently using Go 1.20.12. Does it still work if you use that compiler version? Or does it run into the same problem? It could be that Go 1.21.x fixes some issue that Go 1.20.x has with a syscall on s390x.

How can I make sure that when installing esbuild using npm it compiles using the GO compiler that is specified in the environment?

That's not how esbuild's npm package works. The npm package contains pre-compiled esbuild executables and doesn't expect the system to have a go compiler or attempt to make use of the system's go compiler at all.

[AlvinChacko]

I was using go version go1.18.10 zos/s390x because that was already available. I can check with 1.20 and 1.21 see if that makes a difference.

[AlvinChacko]

Ok here are my findings:

I didnt have esbuild in my devDependencies so by default when @angular-devkit/build-angular@15.2.10 is installed it defaults to the esbuild-wasm@0.17.8 and my compiled version of esbuild was not getting picked up. Is there a way to debug this further by setting the ESBUILD_BINARY_PATH so esbuild-wasm picks it up?

When I added esbuild@0.17.8 in my devDependencies, I get the error Error: Unsupported platform: os390 s390x BE and the npm install command fails.

If I point the ESBUILD_BINARY_PATH to the esbuild I compiled using z/OS GO and then do the npm install Im able to go past the above error and build everything just fine.

What do you think is the right approach here is?

[AlvinChacko]

Any updates on how this can resolved other than keeping a copy of the binary file?