Targeting wasi-sdk?

[thegecko]

Cool library and a great idea.

There's quite a lot of interest around WASI and implementing hosts in various ecosystems which can offer up a posix-like filesystem to wasm binaries.

Have you considered how easily this would port to be built using wasi-sdk?

[jprendes]

I haven't tried, but would be an interesting exercise. I think wasi-sdk has implements less posix apis that emscripten, but it should be fine as long as LLVM doesn't try to use them

[thegecko]

It seems emscripten is a known host to llvm, but wasi isn't so I've had to hack it about quite a bit. Still no luck, but wanted to make sure this hasn't been tried already.

[jprendes]

Did you try it? Do you have any logs or error message?

[thegecko]

I'm fiddling about over here:

https://github.com/thegecko/llvm-box

It's an isolated llvm-box fork from your repo targeting arm architecture. The main branch has an emscripten wasm building (similar to the PR I just opened) and I'm now working on switching to wasi-sdk on this branch:

https://github.com/thegecko/llvm-box/tree/split

Changes have included:

• using the wasi-sdk docker image
• using the wasi-sdk toolchain file as described here
• forcing the platform to be unix (-DUNIX=True) to get past this check

So far it configures, but the build is failing with many errors, see: https://github.com/thegecko/llvm-box/actions/runs/6651894922/job/18074801267

I've disabled the patching to get a box release and am just focussing on building a single tool initially.

[jprendes]

Hm, I see. In that particular case LLVM is using signals. Emscripten has special handling for signals, while it seems that wasi-sdk doesn't define them at all. That's what I meant by I think wasi-sdk has implements less posix apis that emscripten. You could try borrowing Emscripten's implementation of raise and replace the JS bits with C code.

[Johnn333]

It seems emscripten is a known host to llvm, but wasi isn't so I've had to hack it about quite a bit. Still no luck, but wanted to make sure this hasn't been tried already.

This could be of interest if you are just looking at building LLVM tools. https://github.com/binji/wasm-clang. The talk linked in the README is a good watch, quite a long one, but gives a good overview of the process. Things may have changed since 2019, but I imagine still quite similar.

[Johnn333]

So far it configures, but the build is failing with many errors, see: https://github.com/thegecko/llvm-box/actions/runs/6651894922/job/18074801267

Not sure if you have tried building with these flags yet: -D_WASI_EMULATED_SIGNAL. Could get you to the next error :)

[thegecko]

Not sure if you have tried building with these flags yet: -D_WASI_EMULATED_SIGNAL. Could get you to the next error :)

I did:

https://github.com/thegecko/llvm-box/actions/runs/6653847540/job/18080685396

Led to a missing machine/endian.h file.

[Johnn333]

Just had a little poke around, wasi-sdk does ship with an endian.h file. #include should be fine, based on paths, but im not convinced its actually triggering. #include <machine/endian.h> is failing because endian.h doesnt exist in /machine (nothing does). This code is triggering. We may need a WASI entry here

[thegecko]

Thanks @Johnn333 , now back to more errors with signals. I assume these are a fundamental requirement in llvm?

https://github.com/thegecko/llvm-box/actions/runs/6659533030

[jprendes]

I think the problem is that you need to define _WASI_EMULATED_SIGNAL in the C++ code, not as a cmake argument. https://github.com/thegecko/llvm-box/blob/split/build-wasi.sh#L43C1-L43C37

Try adding it to CXXFLAGS here: https://github.com/thegecko/llvm-box/blob/split/build-wasi.sh#L24

[thegecko]

Thanks, I wondered if that was the case, I'll give it a go!

[thegecko]

Still getting signal errors 😕

[Johnn333]

Still getting signal errors 😕

Average day trying to build LLVM from source 😤. Could be a case that the WASI signal implementation isn't enough for the support LLVM is expecting, although the errors do look different now. I think one thing I would try is removing any reference to pthreads, It does complicate matters quite a bit, as I know for the clangd implementation we have over here (Chances are it won't work unless you have your browsers dev tools open). [40/2598] files compiled isn't looking too great at this point though :( , could be a long road ahead. Only so much flag changes can do.

[thegecko]

I struggled without pthread support, it fails to configure.

Which flags did you use to turn this off?

[Johnn333]

With Emscripten it was more so a case of not building with the -pthread flag. I think WASI is adding this in somewhere but looks like you have resolved that. You may also need to add this cmake flag: -DLLVM_ENABLE_THREADS=OFF, no saying this actually fixes anything even once past the configuration :)

[thegecko]

You may also need to add this cmake flag: -DLLVM_ENABLE_THREADS=OFF

Yeah, that's already set:

https://github.com/thegecko/llvm-box/blob/split/build-wasi.sh#L36

[Johnn333]

Interesting, maybe the pthread toolchain file is adding more than just the -pthread flag, is cloning the repo and removing just that specific flag reasonable? or does the Docker build not rely on a git repo? I think it may be better time spent to fix the current issues anyway and only bother with threading if an issue involving it comes up? It was just a thought.

[jprendes]

It looks like LLVM's cmake is trying to fing pthreads even if it isn't going to use it because its disabled by LLVM_ENABLE_THREADS.