search

melange-re / melange

A mixture of tooling combined to produce JavaScript from OCaml & Reason
https://melange.re
Other
828 stars 50 forks source link

Testing for regression #71

Closed idkjs closed 1 year ago

idkjs commented 3 years ago

Tested a project to see if it would build with this fork.

bs-react-dropzone on  master [!?] is 📦 v0.6.0 via ⬢ v14.15.4 took 41s 704ms
 [I] ➜ yarn
yarn install v1.22.10
warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.
[1/4] 🔍  Resolving packages...
success Already up-to-date.
$ npm run clean && npm run build

> bs-react-dropzone@0.6.0 clean
> esy bsb -clean-world

> bs-react-dropzone@0.6.0 build
> esy bsb -make-world

File "/Volumes/SSD/Github/bs-react-dropzone/_build/default/src/ReactDropzone__Utils.ml", line 1, characters 11-13:
1 | let unary (fn: 'a) : 'a = [%raw "(a) => fn(a)" ];;
               ^^
Warning 27 [unused-var-strict]: unused variable fn.
✨  Done in 4.52s.

It should probably be done systematically. Per our brainstorm, basically, build a bunch of big projects with this stack and see what happens. How should we go about that?

anmonteiro commented 3 years ago

cc @Mathspy who has some ideas for this.

Mathspy commented 3 years ago

Hey! Yep, I have quite a bit of ideas here highly inspired by Rust's crater

So I think first thing we have to decide is scope, to me the scope of the initial work is to verify and ensure compatibility with both "pre-ReScript BuckleScript" and ReScript itself The way I'd approach this is by scraping GitHub's public repositories for repositories that have Reason, ReScript or OCaml AND a bsconfig.json. This is approximately the largest open source data set we can get our hands on that we can attempt to verify Melange against. Now this would likely be daunting so we can save this dataset and order it by stars and pick the top ~20 and a couple of others we are confident about or have already tested to start and crawl our way to the full set eventually. If something is known to be broken it can be put into a block list (potentially with a commit hash) where we don't attempt to build it (potentially until commit hash changes)

The way I'd do the actual verification is by attempting to build with the desired version of bs-platform (from package-lock.json or yarn.lock; also probably want to ignore at this step bs-platform versions older than v8) and comparing the output with the one from adding an esy.json with latest version of Melange and running esy. Manually comparing some of this output might give us more clues about the new warnings and whether we might want to disable some of them (if they are too common and unreasonably pedantic for instance)

I could get started on this as soon as you give me the green light, especially since it's weekend ❤️

For reference:

jfrolich commented 3 years ago

Great idea @Mathspy, we can also try to acquire some production closed source code bases to test against in private to look for regressions. I already did this exercise with @anmonteiro for our codebase. This because open-source is mostly library code and production codebases can be quite expansive that might have more probability for edge cases.

johnhaley81 commented 3 years ago

I worked with @anmonteiro a but to try and get our private repo to work as well. Would definitely be game for sharing that and making sure that it build.

anmonteiro commented 3 years ago

@Mathspy that sounds great. let's put a very simple prototype together and go from there.

I'm interested to know which PPXes out there get broken as well. I've already translated some of them to ppxlib which makes it work with both Melange and ReScript.

idkjs commented 3 years ago

@anmonteiro I would love to see the before and after diff of a translated ppx. Any of those ppx's public?

anmonteiro commented 3 years ago

@idkjs

anmonteiro commented 2 years ago

This would have been useful to have today. I'd love to know whether I introduced ecosystem regressions with #284

anmonteiro commented 1 year ago

This is a bit out of scope now: