lrascao / rebar3_gpb_plugin

A rebar3 plugin for automatically compiling .proto files using the gpb protobuf compiler
MIT License
54 stars 37 forks source link

Shift workflow Erlang version from 23..25 to 24..26 #165

Closed tomas-abrahamsson closed 1 year ago

tomas-abrahamsson commented 1 year ago

In the github workflow actions, run on Erlang versions 24..26

Creating the PR as a draft for now to get the workflow actions to run, to see status.

tomas-abrahamsson commented 1 year ago

Hmm... I'm not entirely sure how to best solve this situation. In Erlang 26, "Dialyzer has enabled (by default) warnings about unknown types and functions." in its PR 6822. So Dialyzer needs knowledge about rebar types and functions to be able to check rebar3_gpb_plugin.

Adding rebar as a dependency is probably (?) wrong. Another way to go could perhaps be to (in the github workflow files) add a step to build a plt for rebar, and add it as an extra plt in this project's rebar.config. Maybe it should be a helper script somewhere so that it would be easy to run both locally before commit as well as from a github workflow. What version of rebar to use when build that plt? The latest perhaps.

lrascao commented 1 year ago

Adding rebar as a dependency is probably (?) wrong. Another way to go could perhaps be to (in the github workflow files) add a step to build a plt for rebar, and add it as an extra plt in this project's rebar.config. Maybe it should be a helper script somewhere so that it would be easy to run both locally before commit as well as from a github workflow. What version of rebar to use when build that plt? The latest perhaps.

That would probably work best, the latest rebar might be fine since they're already building against the oldest supported OTP version

tomas-abrahamsson commented 1 year ago

I'll try to take another look at this. (I must confess I had almost forgotten.)

tomas-abrahamsson commented 1 year ago

I tried to include rebar3 in the set of types for dialyzer, but failed. So I added no_unknown to the dialyzer options in rebar.config to suppress issues about unknown functions and types, namely those in rebar3 itself.

Maybe it is still possible to find some way to run dialyzer and include rebar3 too, but I didn't manage to make that happen. Here's a list of attempts that failed for me (various degrees of ugliness):

Adding rebar3 as a dependency (in a profile)

But it doesn't seem like rebar3 itself is published on hex, unless I'm missing something. And it probably makes sense — how would one fetch it?

Putting rebar3 apps into plt_extra_apps

I made a wrapper script that unzips rebar to a temporary directory and synthesizes a rebar.config. In this temporary rebar.config, I tried to add rebar3's apps by specifying them in the plt_extra_apps.

But this fails with what I think are code path issues: beam files of rebar3 apps are present in the escript, and I think they are already loaded into the beam vm. When rebar3 tries to locate the plt_extra_apps the mechanisms to locate them in rebar_prv_dialyzer fails to find them.

I tried to circumvent this by tweaking the Erlang vm's code path in various ways, but I couldn't get it to work. I tried setting the ERL_LIBS environment variable, and I tried to put a path into ERL_FLAGS by setting it to "-pa //ebin" for all the apps in the rebar3 escript.

(Ab)using extra_src_dirs

It looks like the rebar3_prv_dialyzer can pick up files based on the extra_src_dirs config, so with the approach above to unzip to a temporary directory, I tried to specify all ebin subdirs of the rebar3 escript in the temporary rebar.config. But this failed as well. rebar3 complains about duplicate files for some reason...

Duplicates of module bbmustache: /tmp/rebar3_gpb_plugin-r3-W3lBQfx/bbmustache/ebin/bbmustache.beam /tmp/rebar3_gpb_plugin-r3-W3lBQfx/bbmustache/ebin/bbmustache.beam

Copying all rebar3's beam files to the build dir

In the wrapper, I copied all beam files from the unzipped rebar3 to _build/default/lib/rebar3_gpb_plugin/ebin/, then ran rebar3 dialyzer. But even this failed.

Could not scan the following file(s):
  Could not get Core Erlang code for: /usr/local/src/rebar3_gpb_plugin/_build/default/lib/rebar3_gpb_plugin/ebin/rebar_config.beam
  [redacted a lot more rebar3 beam file unzipped from the escript]
tomas-abrahamsson commented 1 year ago

It turned out the no_unknown option is unknown in Erlang 25 and earlier. So turn the rebar.config into a script to be able to include this option conditionally, depending on Erlang version.

Error in dialyzing apps: Unknown dialyzer warning option: no_unknown
tomas-abrahamsson commented 1 year ago

phew all checks finally succeeded.