Support GHCJS compilation

[justinlovinger]

Is your feature request related to a problem? Please describe. GHCJS cannot compile accelerate.

Describe the solution you'd like GHCJS to successfully compile accelerate.

Describe alternatives you've considered Don't use accelerate.

Additional context accelerate currently uses foreign C calls that fail under GHCJS. Additionally, the terminal-size dependency does not compile with GHCJS.

I'm not sure how practical it would be to support compilation under GHCJS, but it would make accelerate an even more backend agnostic framework for computation. With GHCJS, accelerate could run on the web, in addition to CPUs and GPUs.

[tmcdonell]

Could you elaborate a bit on the use case? Who is running [data-parallel] array programs on the web?

Moreover we'd need one of the real backends as well, which means ... converting all of LLVM to javascript so it can run on the web as well? That's insane.

Saying that, I do see that something like a web-assembly backend for Accelerate could maybe be useful. You'd have Accelerate generate the code offline and just incorporate the result into your site, rather than doing all of the compiling & running online...

I could be wrong, so I'm interested to hear what you plan to do here.

[justinlovinger]

I'm developing a series of mathematical optimization and machine learning libraries in Haskell. I already published a PBIL implementation backed by Accelerate, https://github.com/JustinLovinger/optimal-hs-accelerate-pbil, but I have a lot more in development.

I'm also working on a prototype for a web game/teaching aid. It uses mathematical optimization methods like PBIL.

If Accelerate can be compiled with GHCJS, I can use the same code for offline machine learning research, where performance is important, and for web applications. Otherwise, I'll have to either use a different array implementation or duplicate a significant amount of code.

That's just what I'm working on right now. I have a number of ideas and potential research projects that involve using my libraries on the web, so GHCJS support is important to me.

As for backends, I figured the reference implementation could run on the web. At least for my use case, performance on the web isn't as important as code de-duplication.

[tmcdonell]

I have a number of ideas and potential research projects that involve using my libraries on the web, so GHCJS support is important to me.

Would an offline Accelerate → WASM backend work for this use case? (and would it be better?)

I'm just wondering what the easiest/best course here is. As you saw there is a bunch of C code in Accelerate, but maybe that isn't useful in this context and we can just make GHCJS ignore them. At a quick glance I see debugging/profiling hooks and interacting with the terminal, both of which I assume you can live without. I haven't dug any deeper than grep however.

If you want to start working on this we can help you getting a PR merged.

[justinlovinger]

I think you would still need to compile the core accelerate package to WASM to use Accelerate on the web with WASM. Whether or not you can use Accelerate with WASM would come down to whether or not the Haskell to WASM compiler can compile a Haskell package with C code. Last I checked, Haskell WASM compilation is still an area of active development, and there isn't a defacto solution. WebGHC mentions compiling C code and LLVM. It may work.

It was also my impression that the C code isn't essential. It may be possible to refactor Accelerate to move the C code for debugging and pretty printing into separate packages. However, I'm not familiar enough with Accelerate or the Haskell FFI to know for certain.

I think removing the C code from Accelerate is a more appropriate project for someone more familiar with Accelerate and the Haskell FFI.

If I end up trying to compile Accelerate to WASM, I'll let you know what the results are, but I'm not sure if Haskell WASM compilation is mature enough.

[tmcdonell]

We wouldn't be compiling Haskell to WASM, the pathway would be Accelerate → LLVM → WASM. The result would be part of the static site data, and the question was whether that would be enough. Regardless, it sounds like overkill for your use case, so just getting accelerate to compile in ghcjs and using the interpreter is the path forward.

I don't have the time or inclination to work on this (I've never used ghcjs and have no idea how to set it up/use it) but am happy to help you get a PR merged. Here's a rough checklist of things that need to be done.

• [ ] Get accelerate compiling under ghcjs. You should be able to just #ifdef out the unsupported parts.
• [ ] Add build & usage instructions, including notes on any unsupported configurations/features
• [ ] Add ghcjs build(s) and tests to the CI

[justinlovinger]

I tried compiling Accelerate to Wasm using wasm-cross. It failed for similar reasons, broken dependencies. terminal-size fails to build with error src/System/Console/Terminal/Posix.hsc:24 directive let cannot be handled in cross-compilation mode. cryptonite failed because it's dependency, basement failed with error error "foundation: system: Unknown compiler". There may be other broken dependencies that would reveal themselves when the above are fixed.

I imagine fixing Accelerate to compile with GHCJS would also fix Accelerate to compile to Wasm. However, fixing Accelerate to compile to Wasm may require fewer modifications.

P.S. I have to wonder if all 36 Accelerate dependencies are strictly necessary. I've never seen a Haskell project with more dependencies than Accelerate.

[tmcdonell]

You just have to think bigger, stack has at least 85 direct dependencies.

Anyway, certainly some of those might be removed: we don't require all the features of cryptonite, for example, so I'm open to discussing how the dependencies can be trimmed down. We've previously done this to remove the dependency on lens, for example, to avoid all the extra indirect dependencies that that pulled in.