The ghcvm-hackage system

[rahulmutt]

There are minor differences between GHC and GHCVM and those minor differences must be accounted for to allow libraries on Hackage to compile. To consult with every package maintainer about coming up with the fixes themselves is a time-consuming task as well as waiting for them to release a new version that will contain GHCVM support via a conditional #ifdef __GHCVM_VERSION__ check.

I propose that we:

• introduce a new repo called ghcvm-hackage which contains information on package, version, and how to patch that specific package version so that GHCVM can compile it. (This will literally be a git patch file).
• Add a new feature in cabalvm that will clone the ghcvm-hackage repo locally and keep updating it. Whenever a user invokes cabalvm install [package], the package will be looked up in ghcvm-hackage and the source distribution will be patched with the patch file just before cabalvm starts the build.

The resulting flow:

1. A user tries installing a package and finds that the build fails.
2. He files an issue to the ghcvm-packages repository
3. People who want to help out will:
   • download that package from hackage locally
   • make the necessary changes to fix the build
   • submit a PR with the patch file to the ghcvm-hackage repo
   • the end-user will be able to successfully install the package upon another invocation of cabalvm install

Looking forward to feedback.

[LeanderK]

What are the differences? do you think many packages will be affected? (the threading would be different i guess, since we don't have green threads on the jvm. But that should not change syntax)

[rahulmutt]

I've taken care to make sure the public API of the core packages of is the same as the corresponding package in GHC (for ghc-prim, integer, and base) so the differences will be things like #include of header files that are not available in GHCVM or don't make sense and use of c sources/dependencies. And there may be a need to change the usage of certain primitive operations since the semantics are slightly different from GHC in certain cases.

Take for example, the text package. It uses native C code for utf8 encoding/decoding. It clearly won't compile out-of-the-box, yet it's one of the most frequently used dependencies. And I feel forking every single package is a waste since I see that most of the changes will be minor (diff of 10-20 lines across the package at most).

ghcvm-hackage you can think of as the bleeding-edge port of Hackage libraries for GHCVM. The goal is to send PRs to the upstream packages with the changes placed inside #ifdef __GHCVM__VERSION__ and have the maintainers upload a new version. This won't work for all the packages since they'll have to carry around java files which require GHC Cabal-incompatible changes to the .cabal file (usage of the java-sources: field which GHC Cabal will probably die on).

We will face no issues with pure Haskell packages since they probably won't have any #ifdefs other than for maintaining compatibility with Haskell dependencies. But it just so happens that many pure-haskell libraries depend on libraries like text for the sake of defining instances.

[rahulmutt]

The main goal of doing this is to prevent from forking a significant chunk of Hackage when all we're doing is changing 10-20 lines.

Oh, and I also forgot to mention those packages that depend on template-haskell will have problems as well.

[puffnfresh]

:+1:

[LeanderK]

well, it seems a bit hacky (what happens with the patch when the version changes?), but i think the first objective is to get it to work at all.

[rahulmutt]

My biggest priority now is getting people to start using GHCVM and getting a good number of libraries ported ASAP would help in that effort. If people start building sophisticated applications, we'll get an idea of where we need to optimise, fix any bugs we wouldn't have found otherwise, and evolve the project based on the use cases. In order for people to do that, they need good libraries that cover all the standard tasks of modern software development like calling REST APIs, connecting to the cloud, connecting to SQL & NoSQL databases, etc.

[rahulmutt]

Implemented with #60