search

melange-re / melange

A mixture of tooling combined to produce JavaScript from OCaml & Reason
https://melange.re
Other
828 stars 50 forks source link

Proposal: ability to compile dependency code outside of node_modules #75

Closed jfrolich closed 3 years ago

jfrolich commented 3 years ago

ReScript will compile dependencies and place the bs.js files alongside the .re/.res files. This means that even though we can check-in the bs.js files of our own project. We still need to run the compiler to generate the bs.js files in the dependencies (they usually don't ship compiled).

Because of this, we need to run the compiler on our CI. In our case, we run a CI in GitHub actions to verify correctness, for our website, but also for our react-native app. Given the tooling is much heavier, I don't think I can justify the whole esy tooling as it will dramatically increase compile times (or we have to do some advanced caching everywhere).

One solution would be to be able to compile the dependencies somewhere outside of node_modules and "promote" theme to node_modules in a separate build step that is also included in a postinstall script (so when a project is compiled and delete my node_modules folder, I get the same "state" after a reinstall.

Perhaps my solution is not the best one. Another solution would be to have portable binaries. Any thoughts?

anmonteiro commented 3 years ago

I think this is generally related to #33.

Note, however, that this is already possible. all your JS files end up in _build under the same paths, along with all the other build artifacts. is that not enough for you?

jfrolich commented 3 years ago

@anmonteiro: Ah yeah that might work! I can basically just check-in the node_modules directory of the _build folder and then rsync that on the CI!

anmonteiro commented 3 years ago

Yup, please report back! Note that there will be lots of *.cm* files in there too.

jfrolich commented 3 years ago

Yup, please report back! Note that there will be lots of *.cm* files in there too.

Yeah just set up .gitignore rules to just keep the .bs.js

EduardoRFS commented 3 years ago

@jfrolich I'm not sure the caching would be too much, essentially you just need to cache the .esy folder like the esy action on github does, and that should give you a CI that takes ~40s + build time.

Of course there is also caching of node_modules, which hopefully can be part of the .esy if we get pnp to work.

jfrolich commented 3 years ago

@EduardoRFS: We have to deal with AppCenter (ios/android CI pipeline) and Vercel (for web). I'll have to see how easy it is to add caching. Just checking in the .bs.js artifacts is an easy lightweight path for us, that might make sense in some scenarios. Pnp can be problematic with some javascript tooling like react-native I think.

anmonteiro commented 3 years ago

Sound like this is possible today by using the artifacts in _build. let me know if that's not the case, and we can revisit.

EduardoRFS commented 3 years ago

@jfrolich yeah you can still cache the .esy and node_modules, but it is going to be a heavier pipeline.

But yeah the assets are in the _build now, so they just moved a bit around.

jfrolich commented 3 years ago

I fixed it with checking in the _build output of *.bs.js files in the node_modules subfolder and rsyncing it to the root node_modules in a postinstall script. Pretty great, no build tooling required!