Things I'd need for my project

[TheSpyder]

I've successfully compiled the native portion of my project using bsb-native, and I have some feedback on what it would take to use it over jbuilder.

I'm working on a project that compiles to JS, but I've clearly segregated the pure ReasonML core from a very thin JS layer for working with the browser. The idea here is that my tests are written in native for performance, and they don't need to test the JS layer; the pure core allows near-complete testing of the project.

The source code in my project is structured like this:

/
- src
  - main
    - stuff here
  - test
    - stuff here

Of the files in /src/main, only a few make use of JavaScript features and they're all in a single folder. /src/test meanwhile is 100% native, making extensive use of ounit with some extlib and ocaml-re as well. Jbuilder works fairly well for me but compiling twice is annoying.

To make this work with bsb-native, I had to use the ocamlfind-trial branch (I tried it a couple of weeks ago). I added my test folder(s) to the source tree, removed the JS specific folder, and then added this:

  "entries": [{
    "kind": "native",
    "main": "AllTests"
  }],
  "ocamlfind-packages": ["ounit", "extlib", "re", "re.perl"],

It works really well, aside from being a larger binary than Jbuilder produced, but of course now I'm stuck between two worlds:

• With the native config, I can't compile my JS layer (the code has no native equivalent).
• With a JS config, I can't compile my tests.

I could potentially run a hacky setup where my Makefile swaps between two bsconfig files, but here's my ideal setup:

• Multiple objects in the entries array, so I get both native and JS output with a single bsb instance. This is because once my tests pass I want to use the code in JS land.
• Source folders default to being available to all backends as they are today, but there is a way flag to tag a source folder as being specific to either native or JS (perhaps an array matching to the kind value in the entries objects?)
• run my tests as the last task in ninja (this isn't a bsb-native specific requirement, just thought I'd mention it)

[bsansouci]

Hey I just did some benchmarks of master vs the ocamlfind-trial branch. It seems like the former's only about 7% faster, which is probably not even statistically significant. I'm gonna merge it in!

Here are the more results (just so I paste them somewhere):

Building Reprocessing

=========== bsb-native + ocamlfind

Bytecode: 2.612 2.486 2.846 1.835 2.144 1.869 1.924 2.051 1.919 1.815 2.167 1.853 1.768 1.974 2.081 average = 2.0896

Native: 3.544 3.116 3.113 3.250 3.041 3.152 3.270 3.105 3.542 2.855 3.882 3.837 3.245 3.288 2.919 average = 3.2772666667

========== bsb-native

bytecode 2.809 1.866 1.674 1.935 1.810 1.722 1.950 1.803 1.724 1.999 2.342 1.908 1.967 1.934 1.651 average = 1.9396 (7.1% faster)

native 3.600 3.069 3.141 2.787 3.004 3.242 2.904 2.853 3.148 3.070 2.923 3.167 2.992 3.137 2.784 average = 3.0547333333 (6.7% faster)

[bsansouci]

I merged the ocamlfind-trial branch into master! Master also builds everything in the entries array following a command line flag called -backend. So to compile all of the things in the array that have "kind": "bytecode" you'd run bsb -make-world -backend bytecode.

The only thing missing for you is that when you'll build to JS, bsc will fail on your tests. That's because bsb-js build everything in your project without trying to be smart. I'm not sure of the best way to fix this... Either:

• "be smart" and only build what's used starting at a root module or
• implement a way to tag folders in the dir structure as "js-only" (like allowed-build-kinds for dependencies which will tell bsb whether or not a dep can be build for an architecture).

@bobzhang did you have plans to support building only the files referenced from a root module in bsb/bsc? The problem at hand is that @TheSpyder has native code and js code in the same codebase and we'd like to make sure the native-specific code does no get compiled to js (and vice versa). For context I avoided the problem and separated my native/js specific code into full-on deps (ReasonglNative, ReasonglWeb etc...) and I made a flag called allowed-build-kinds which tells bsb if a specific dep can be built to the specific target (so ReasonglWeb's allowed-build-kinds only contains "js").

[TheSpyder]

7% difference is relative ;)

After running a few iterations my project + tests takes 3.0-4.0 seconds to compile with jbuilder, bsb-native does it in 2.0-2.5.

That's using the ocamlfind branch after adding -backend native on the command line, and updating bsconfig to the new ocamlfind-dependencies key. Master isn't working for me. I'll get to why in a moment.

"be smart" sounds easiest to deal with, but sort of breaks the "build everything" nature of bucklescript (imagine someone migrating a big codebase to reason one file at a time, there won't be an entry point). I think adding a kinds array property to the directory spec is clean, and most importantly won't impact any developer who doesn't need it. I also like that idea more than allowed build kinds which I'd have to set on every other directory rather than just the one that needs it (tests).

So. Why is master broken for me. On the first attempt it refused to do -make-world correctly combined with -backend native, but that was before I fixed the ocamlfind dependencies. I'm now getting this:

Error: Files /Users/spyder/.opam/4.02.3+buckle-master/lib/ocaml/unix.cmxa
       and /Users/spyder/.opam/4.02.3+buckle-master/lib/ocaml/unix.cmxa
       both define a module named Unix

I double checked by switching back to the ocamlfind-trial branch and it works with the same config.

[bsansouci]

@TheSpyder The issue you mentioned here should be solved on master. I'm working on getting you the folder-level backend kind.

[TheSpyder]

Folder backend kind is mostly working, but doesn't expand subdirs correctly yet. Looking good!

The rest (multiple backends simultaneously, running my tests) are perhaps things that bsb isn't designed to achieve so I'm happy just using my Makefile for them.

[bsansouci]

Ok everything here should be fixed :)

[TheSpyder]

Cool. Sadly I had to stop using bucklescript for my production code, but I'll definitely give it a go sometime :)

[bsansouci]

Oooh what! Why?

[TheSpyder]

I have a menhir-generated parser, and the amount of mutual recursion it uses requires trampolines on browsers without TCO (i.e. everything except Safari). Chrome in particular has a very low recursion ceiling before it runs out of stack space. JSOO has trampolines, could be a while before Bucklescript does.

[bsansouci]

Oh right I remember. Well darn. We’ll get it one day :)