rebar3 plugin

[seanhinde]

As promised in #19 I've had a go at a rebar3 compiler plugin

So far this is standalone and very basic, relying on an existing caramel installation and very new rebar3 features

I'm not sure about rebar3 support for git submodules, but I'm happy for you to host this as another repo under AbstractMachinesLab if and when you think it's ready.

[seanhinde]

Also just realised it doesn't handle process.erl. That one I had copied manually into my rebar3 project. I'll iterate a bit more..

[seanhinde]

Added an example project to the plugin repo.

It's not at all clear to me how to handle the caramel runtime modules. Maybe in the direction of lfe and have the plugin install its own local caramel and take care of everything.

Or this plugin is a short term effort and crane will become the one true way for all caramel projects.

Thoughts? Preferences?

[leostera]

Hej @seanhinde sorry for the delay 🙏🏼 been pretty swamped at work. This looks phenomenal 🚀 and thanks so much for putting time into it.

I'd very much like to have this plugin available since crane may take some time to get to a releasable state, and rebar3 is already here.

I think the plugin will have to detect the arch you're running on and download the right release file from Github for it to work nicely -- then you have control over the entire thing. And since right now it means just compiling an extra process.erl, it doesn't seem like that big of a thing. But let me know :) I haven't written rebar plugins yet.

We'll have to figure out how to compile transitive dependencies, and whether rebar3 supports batch compilation of files (we'll need this since we need to support compiling a.ml, b.ml, c.ml in order or as a unit), and whether it makes sense for the Stdlib to also pack .beam files (or the appropriate .app). Let me know what you think about these.

But otherwise I'll try to spend some time with it tonight and give you more feedback. Thanks again 🙌🏼

[leostera]

Hej @seanhinde I tried to use the plugin and while I got a clean rebar3 compile without errors it didn't seem to pick up the .ml file I had around. This was on the example project included.

I essentially:

• git cloned
• ran rebar3 compile on the example project, saw that caramelc wasn't on path
• made sure caramelc was on path
• ran rebar3 clean, compile on the example project
• run tree -9, and I see this:

.
├── apps
│   ├── beam
│   │   └── src
│   │       ├── beam.app.src
│   │       └── process.erl
│   └── caramel_readme
│       └── src
│           ├── caramel_readme_app.erl
│           ├── caramel_readme.app.src
│           ├── caramel_readme_srv.ml
│           └── caramel_readme_sup.erl
├── _build
│   └── default
│       ├── lib
│       │   ├── beam
│       │   │   ├── ebin
│       │   │   │   ├── beam.app
│       │   │   │   └── process.beam
│       │   │   ├── include -> ../../../../apps/beam/include
│       │   │   ├── priv -> ../../../../apps/beam/priv
│       │   │   └── src -> ../../../../apps/beam/src
│       │   └── caramel_readme
│       │       ├── ebin
│       │       │   ├── caramel_readme.app
│       │       │   ├── caramel_readme_app.beam
│       │       │   └── caramel_readme_sup.beam
│       │       ├── include -> ../../../../apps/caramel_readme/include
│       │       ├── priv -> ../../../../apps/caramel_readme/priv
│       │       └── src -> ../../../../apps/caramel_readme/src
│       └── plugins
│           └── rebar3_caramel
│               ├── ebin
│               │   ├── rebar3_caramel.app
│               │   ├── rebar3_caramel.beam
│               │   └── rebar3_caramel_compiler.beam
│               ├── examples
│               │   └── caramel_readme
│               │       ├── apps
│               │       │   ├── beam
│               │       │   │   └── src
│               │       │   └── caramel_readme
│               │       │       └── src
│               │       ├── config
│               │       │   ├── sys.config
│               │       │   └── vm.args
│               │       ├── LICENSE
│               │       ├── README.md
│               │       └── rebar.config
│               ├── LICENSE
│               ├── README.md
│               ├── rebar.config
│               └── src
│                   ├── rebar3_caramel.app.src
│                   ├── rebar3_caramel_compiler.erl
│                   └── rebar3_caramel.erl
├── config
│   ├── sys.config
│   └── vm.args
├── LICENSE
├── README.md
├── rebar.config
├── rebar.lock
└── tree_stuff

Let me know if I'm doing something wrong here or if I missed any other steps

[leostera]

Ah, nevermind, I had modified the .ml file and forgot about it. It fails silently when there's a type-checking error at the moment but otherwise it does seem to work 🙌🏼 🚀

[seanhinde]

Nice. I should find some more time to work on it soon. I also have a job, but have half an eye on this being useful there

[seanhinde]

A couple of immediate challenges:

Erlang's os:cmd/1 ignores the exit code of the command it is running. Solutions to this are a bit painful (there is a project erlexec, or we implement our own using the underlying primitives). If the output of caramelc is pretty regular another option is just to parse its output. I didn't look too closely.

As far as I can tell rebar3 doesn't provide a way to download architecture specific files as a simple dependency.

Both workaroundable.

For module dependencies the rebar plugin has a mechanism. My first idea was to grab the output of caramelc sort-deps and declare that every file earlier in the list than the current file is a dependency. Not sure if that's exactly right though..?

[leostera]

I think we could use os:type/0 to pick the OS for now and see how far that takes us.

For running caramelc we can probably use a port that we open/close. That should give us all the info we need.

Regarding the dependencies, I'd expect the rebar3 dependency tree to automatically build things in the right order, but I may be wrong.

What do you say we move the repo to this org, I'll give you full access, and we can start shaping it up there?

[seanhinde]

Sounds good to me. If you invite me as a user I can probably just import what I have

[seanhinde]

Or you can probably clone my repo and import it as a new project here. I'm not that familiar with how GitHub roles work

[leostera]

Hi! I think I sent you an invite as an outside collaborator but it don't really know what's what in that world. I'll look into it, but it seems that you should be able to go to settings and use the "Transfer ownership" button to get the process started from your end as well.

In the meantime I'm looking into making a mix plugin as well.

[seanhinde]

Nice on the mix front. I fixed the os:cmd issue - rebar3_utils:sh/1 does the right thing.

to trigger a compile failure message it would be good if caramelc will do exit 1 if it finds errors

[leostera]

Good find! That should be the case at the moment, did you find any errors for which it doesn't exit with != 0 ?

Btw, if you're on discord, I started a small server where we can keep the communication more fluid: https://discord.gg/SRpjUE99

[seanhinde]

I have raised an issue on rebar3 GitHub asking about their built in dependency handling. In theory it should be perfect for us, but I can't get it to work

[leostera]

Closing this since we have the separate repository now: https://github.com/AbstractMachinesLab/rebar3_caramel