rebar3 compile fails on the first run (but succeeds after that)

helanhalvan commented 3 years ago

Environment

Add the result of rebar3 report to your message:

$ rebar3 report compile
Rebar3 report
 version 3.14.3
 generated at 2021-02-18T11:27:49+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: compile
Entered as:
  compile
-----------------
Operating System: x86_64-unknown-linux-gnu
ERTS: Erlang/OTP 22 [erts-10.7.2.7] [source-21c03488f9] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]
Root Directory: /erlpath/lib/erlang
Library directory: /erlpath/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.10.0
certifi: 2.5.2
cf: 0.3.1
common_test: 1.18.2
compiler: 7.5.4.3
crypto: 4.6.5.2
cth_readable: 1.4.9
dialyzer: 4.1.1
edoc: 0.11
erlware_commons: 1.3.1
eunit: 2.4.1
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 3.19.3
inets: 7.1.3.3
kernel: 6.5.2.1
providers: 1.8.1
public_key: 1.7.2
relx: 4.2.0
sasl: 3.4.2
snmp: 5.5.0.4
ssl_verify_fun: 1.1.6
stdlib: 3.12.1
syntax_tools: 2.2.1
tools: 3.3.1

-----------------
Escript path: /erlpath/bin/rebar3
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref 

...

Verify whether the version of rebar3 you're running is the latest release (see https://github.com/erlang/rebar3/releases)

$rebar3 --version rebar 3.14.3 on Erlang/OTP 22 Erts 10.7.2.7

If possible, include information about your project and its structure. Open source projects or examples are always easier to debug. If you can provide an example code base to reproduce the issue on, we will generally be able to provide more help, and faster.

To replicate, create a new rebar3 project with rebar3 new app rebar_core_crash replace rebar.config with:

{deps, [ {aleppo, {git, "https://github.com/inaka/aleppo.git", {tag, "1.1.1"}}}
           , {elvis, {git, "https://github.com/inaka/elvis.git", {tag, "0.4.2"}}}
           ]}.

Note that this is an old elvis version interacting with the latest aleppo, and they might not be compatible. This issue goes away with elvis 1.0.0, not sure why, but I still think this is worth reporting as I don't think this problem is caused by the elvis version, merely exposed by this combination of dependencies.

Current behaviour

So, if you run rebar3 compile it will crash the first time, output below. However if you run the same command again, it will complete successfully.

The first time output:

DIAGNOSTIC=1 DEBUG=1 rebar3 compile
===> Expanded command sequence to be run: [app_discovery,install_deps,lock,compile]
===> Running provider: app_discovery
===> Found top-level apps: [rebar_core_crash]
    using config: [{src_dirs,["src"]},{lib_dirs,["apps/*","lib/*","."]}]
===> Running provider: install_deps
===> Verifying dependencies...
===> Fetching aleppo (from {git,"https://github.com/inaka/aleppo.git",{tag,"1.1.1"}})
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: git --version

===>    opts: []

===> Port Cmd: git --version
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: git clone  https://github.com/inaka/aleppo.git .tmp_dir199294610393 -b 1.1.1 --single-branch

===>    opts: [{cd,"/tmp"}]

===> Port Cmd: git clone  https://github.com/inaka/aleppo.git .tmp_dir199294610393 -b 1.1.1 --single-branch
Port Opts: [{cd,"/tmp"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/aleppo

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/aleppo
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Moving checkout "/tmp/.tmp_dir199294610393" to "/tmp/rebar_core_crash/_build/default/lib/aleppo"
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: mv /tmp/.tmp_dir199294610393 /tmp/rebar_core_crash/_build/default/lib/aleppo

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: mv /tmp/.tmp_dir199294610393 /tmp/rebar_core_crash/_build/default/lib/aleppo
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Fetching elvis (from {git,"https://github.com/inaka/elvis.git",{tag,"0.4.2"}})
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: git clone  https://github.com/inaka/elvis.git .tmp_dir436787507460 -b 0.4.2 --single-branch

===>    opts: [{cd,"/tmp"}]

===> Port Cmd: git clone  https://github.com/inaka/elvis.git .tmp_dir436787507460 -b 0.4.2 --single-branch
Port Opts: [{cd,"/tmp"},
            exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/elvis

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/elvis
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Moving checkout "/tmp/.tmp_dir436787507460" to "/tmp/rebar_core_crash/_build/default/lib/elvis"
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: mv /tmp/.tmp_dir436787507460 /tmp/rebar_core_crash/_build/default/lib/elvis

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: mv /tmp/.tmp_dir436787507460 /tmp/rebar_core_crash/_build/default/lib/elvis
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Fetching egithub v0.5.5
===> Version cached at /home//.cache/rebar3/hex/hexpm/packages/egithub-0.5.5.tar is up to date, reusing it
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/egithub

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: rm -rf /tmp/rebar_core_crash/_build/default/lib/egithub
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Moving checkout "/tmp/.tmp_dir661238928902" to "/tmp/rebar_core_crash/_build/default/lib/egithub"
===> sh info:
    cwd: "/tmp/rebar_core_crash"
    cmd: mv /tmp/.tmp_dir661238928902 /tmp/rebar_core_crash/_build/default/lib/egithub

===>    opts: [{use_stdout,false},abort_on_error]

===> Port Cmd: mv /tmp/.tmp_dir661238928902 /tmp/rebar_core_crash/_build/default/lib/egithub
Port Opts: [exit_status,
            {line,16384},
            use_stdio,stderr_to_stdout,hide,eof,binary]

===> Evaluating config script "/tmp/rebar_core_crash/_build/default/lib/egithub/rebar.config.script"
===> Uncaught error in rebar_core. Run with DIAGNOSTIC=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: {badmatch,error}
===> Stack trace to the error location:
[{rebar_prv_install_deps,update_seen_dep,9,
                         [{file,"/home///rebar3/src/rebar_prv_install_deps.erl"},
                          {line,278}]},
 {lists,foldl,3,[{file,"lists.erl"},{line,1263}]},
 {rebar_prv_install_deps,handle_profile_level,7,
                         [{file,"/home///rebar3/src/rebar_prv_install_deps.erl"},
                          {line,183}]},
 {rebar_prv_install_deps,do_,1,
                         [{file,"/home///rebar3/src/rebar_prv_install_deps.erl"},
                          {line,82}]},
 {rebar_core,do,2,
             [{file,"/home///rebar3/src/rebar_core.erl"},
              {line,155}]},
 {rebar3,run_aux,2,
         [{file,"/home///rebar3/src/rebar3.erl"},{line,181}]},
 {rebar3,main,1,
         [{file,"/home///rebar3/src/rebar3.erl"},{line,66}]},
 {escript,run,2,[{file,"escript.erl"},{line,758}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`

Second time output:

 $rebar3 compile
===> Verifying dependencies...
===> Analyzing applications...
===> Compiling rebar_core_crash

(The diagnostic one is really long and I am not sure how much it will help, the issue should be easy to reproduce anyway.)

Expected behaviour

Not sure, it is possible that these two dependencies are incompatible in some way which should result in some error, but I don't think rebar3 should crash. It is rather difficult to know why this is happening with a larger rebar.config. Also, this is inconistent, either rebar3 compile should fail every time or succeed every time.

ferd commented 3 years ago

One of the thing they changed since then is that the app stopped depending on itself (it defined elvis_shell that would be published as elvis and it fetched elvis as an app under the pkg_name elvis_core). This has now been sorted out, and I have to assume that this is what caused the crash. Just apps named weird compared to what is expected and causing issues when fetching and tagging deps.

paulo-ferraz-oliveira commented 3 years ago

Have you tried rebar3_lint (https://hex.pm/packages/rebar3_lint)? It's elvis_core under the hood and maintained by the same developers. (and @ferd is right, the name changing was unfortunate; it broke some expectations).

For reference:

https://github.com/inaka/elvis is the CLI tool (it depends on elvis_core),
https://github.com/inaka/elvis_core is the code tool.

@elbrujohalcon mentioned it here: https://github.com/inaka/elvis/issues/491#issuecomment-423183949.

With the plugin you do

rebar3 lint

and you're good to go.

helanhalvan commented 3 years ago

Yes, we are using rebar3_lint. I found this when trying top map out all the dependencies of the project, including transitive ones.

Even if the root issue is in the dependency itself, (as I mentioned in the post, this has been resolved in 1.0.0), the way rebar crashes makes it hard to figure out what is even wrong. Like, looking at the output with DIAGNOSTIC and/or DEBUG as well as the crashdump, there is nothing pointing to elvis, the only thing you get to work with is that something went wrong when handling: /tmp/rebar_core_crash/_build/default/lib/egithub/rebar.config.script which isn't even a direct dependency.

It is also really strange that rebar3 will crash once and then work. Anyway, mostly opened this issue as this happening was really confusing. I'll use elvis 1.0.0+ from now on, and move on with things. Might be that this weird issue is not worth chasing down.

erlang / rebar3