ninenines / erlang.mk

A build tool for Erlang that just works.
https://erlang.mk
ISC License
578 stars 241 forks source link

Failing to build project in official erlang docker image #950

Closed benjamin-bergia closed 2 years ago

benjamin-bergia commented 2 years ago

Hi,

I am trying to run erlang.mk in the official erlang docker image and for some unknown reasons, I can't get it to work. Here is the structure of my project:

.
├── apps
│   ├── eksd
│   └── metrics
├── config
│   ├── sys.config
│   └── vm.args
├── deps
│   ├── bbmustache
│   ├── jsx
│   └── relx
├── Dockerfile
├── ebin
│   └── ...
├── erlang.mk
├── harvester.d
├── Makefile
├── _rel
│   └── harvester_release
├── relx.config
 └── src
    └── ...

The content of my Dockerfile:

FROM erlang:24

WORKDIR /app/src

COPY . .

RUN make distclean \
 && make 

ENTRYPOINT ["/app/src/_rel/bin/harvester_release"]
CMD ["foreground"]

And the result of my build:

❯ podman build .
STEP 1/6: FROM erlang:24
STEP 2/6: WORKDIR /app/src
--> Using cache d7f1298ef8d6471caf73b49a816e1bc004518a7ce0b1db36821d512b6f8dd120
--> d7f1298ef8d
STEP 3/6: COPY . .
--> Using cache dcdf90a102b2bd06c63a7e3022f745d476ee0ed57e4e902b814fb5b10734ae27
--> dcdf90a102b
STEP 4/6: RUN make distclean  && make 
make[1]: Entering directory '/app/src/apps/eksd'
 GEN    clean-app
 GEN    coverdata-clean
make[1]: Leaving directory '/app/src/apps/eksd'
make[1]: Entering directory '/app/src/apps/metrics'
 GEN    clean-app
 GEN    coverdata-clean
make[1]: Leaving directory '/app/src/apps/metrics'
 GEN    clean-app
 GEN    coverdata-clean
 GEN    distclean-tmp
 GEN    distclean-kerl
make[1]: Entering directory '/app/src/apps/eksd'
 GEN    clean-app
 GEN    coverdata-clean
 GEN    distclean-tmp
 GEN    distclean-kerl
 GEN    distclean-deps
 GEN    distclean-ct
 GEN    distclean-plt
 GEN    distclean-edoc
 GEN    distclean-escript
 GEN    distclean-xref
 GEN    cover-report-clean
make[1]: Leaving directory '/app/src/apps/eksd'
make[1]: Entering directory '/app/src/apps/metrics'
 GEN    clean-app
 GEN    coverdata-clean
 GEN    distclean-tmp
 GEN    distclean-kerl
 GEN    distclean-deps
 GEN    distclean-ct
 GEN    distclean-plt
 GEN    distclean-edoc
 GEN    distclean-escript
 GEN    distclean-xref
 GEN    cover-report-clean
make[1]: Leaving directory '/app/src/apps/metrics'
 GEN    distclean-deps
 GEN    distclean-ct
 GEN    distclean-plt
 GEN    distclean-edoc
 GEN    distclean-escript
 GEN    distclean-relx-rel
 GEN    distclean-xref
 GEN    cover-report-clean
 DEP    relx (main)
No beam files found.
Recompile: src/rebar
Recompile: src/rebar_abnfc_compiler
Recompile: src/rebar_app_utils
Recompile: src/rebar_appups
Recompile: src/rebar_asn1_compiler
Recompile: src/rebar_base_compiler
Recompile: src/rebar_cleaner
Recompile: src/rebar_config
Recompile: src/rebar_core
Recompile: src/rebar_cover_utils
Recompile: src/rebar_ct
src/rebar_ct.erl:291:30: Warning: crypto:rand_uniform/2 is deprecated; use rand:uniform/1 instead
%  291|     Random = integer_to_list(crypto:rand_uniform(0, 10000)),
%     |                              ^

Recompile: src/rebar_deps
Recompile: src/rebar_dia_compiler
Recompile: src/rebar_dialyzer
Recompile: src/rebar_edoc
Recompile: src/rebar_erlc_compiler
Recompile: src/rebar_erlydtl_compiler
Recompile: src/rebar_escripter
Recompile: src/rebar_eunit
src/rebar_eunit.erl:282:20: Warning: crypto:rand_uniform/2 is deprecated; use rand:uniform/1 instead
%  282|             Seed = crypto:rand_uniform(1, 65535),
%     |                    ^

Recompile: src/rebar_file_utils
Recompile: src/rebar_getopt
Recompile: src/rebar_lfe_compiler
Recompile: src/rebar_log
Recompile: src/rebar_metacmds
Recompile: src/rebar_mustache
Recompile: src/rebar_neotoma_compiler
Recompile: src/rebar_otp_app
Recompile: src/rebar_otp_appup
Recompile: src/rebar_port_compiler
Recompile: src/rebar_proto_compiler
Recompile: src/rebar_proto_gpb_compiler
Recompile: src/rebar_protobuffs_compiler
Recompile: src/rebar_qc
Recompile: src/rebar_rand_compat
Recompile: src/rebar_rel_utils
Recompile: src/rebar_reltool
Recompile: src/rebar_require_vsn
Recompile: src/rebar_shell
Recompile: src/rebar_subdirs
Recompile: src/rebar_templater
Recompile: src/rebar_upgrade
Recompile: src/rebar_utils
Recompile: src/rebar_xref
Recompile: src/rmemo
==> rebar (compile)
==> rebar (escriptize)
Congratulations! You now have a self-contained script called "rebar" in
your current working directory. Place this script anywhere in your path
and you can use rebar to build OTP-compliant apps.
/app/src
make[1]: Entering directory '/app/src/apps/eksd'
 DEPEND eksd.d
 ERLC   eksd_endpoints.erl
 APP    eksd
/bin/sh: 1: cannot create ebin/eksd.app: File exists
make[2]: *** [../../erlang.mk:5459: ebin/eksd.app] Error 2
make[1]: *** [../../erlang.mk:5198: app] Error 2
make[1]: Leaving directory '/app/src/apps/eksd'
make: *** [erlang.mk:4465: apps] Error 2
Error: error building at STEP "RUN make distclean  && make": error while running runtime: exit status 2

I am probably missing something obvious but I can't put my finger on it.

PS: Both dependencies in the apps/ directory are libs that I bootstrapped with erlang.mk.

essen commented 2 years ago

I've never seen anything like this. I wonder if you can create a small reproducible example that I can run to investigate.

benjamin-bergia commented 2 years ago

I will give it a try.

benjamin-bergia commented 2 years ago

Ok so far:

Crash dump is being written to: erl_crash.dump...done make: *** [erlang.mk:7539: relx-rel] Error 1 The command '/bin/sh -c make distclean && make' returned a non-zero code: 2

- Tried with both podman and docker. Same problem.
- Can reproduce the error by bootsrapping a new project + release. Adding the following `Dockerfile` and running `docker build .`:
```bash
mkdir test
cd test
wget https://erlang.mk/erlang.mk
make -f erlang.mk bootstrap bootstrap-rel SP=2
FROM erlang:24

WORKDIR /app/src

COPY . .

RUN make distclean \
 && make 

Result:

❯ docker build .
Sending build context to Docker daemon  287.7kB
Step 1/4 : FROM erlang:24
24: Pulling from library/erlang
e756f3fdd6a3: Pull complete 
bf168a674899: Pull complete 
e604223835cc: Pull complete 
6d5c91c4cd86: Pull complete 
2cc8d8854262: Pull complete 
5f7da158a354: Pull complete 
c340b84be241: Pull complete 
7bfbc43914eb: Pull complete 
Digest: sha256:352f22c1912babe4a3230c391eff1741cd64637ef8c6c8671481661c86384604
Status: Downloaded newer image for erlang:24
 ---> eb5cce096456
Step 2/4 : WORKDIR /app/src
 ---> Running in 64ce8b56782b
Removing intermediate container 64ce8b56782b
 ---> 39a06766c70f
Step 3/4 : COPY . .
 ---> bb6b71b3b248
Step 4/4 : RUN make distclean  && make
 ---> Running in fd4d40eff93f
 GEN    clean-app
 GEN    coverdata-clean
 GEN    distclean-tmp
 GEN    distclean-kerl
 GEN    distclean-deps
 GEN    distclean-ct
 GEN    distclean-plt
 GEN    distclean-edoc
 GEN    distclean-escript
 GEN    distclean-relx-rel
 GEN    cover-report-clean
 DEP    relx (main)
No beam files found.
Recompile: src/rebar
Recompile: src/rebar_abnfc_compiler
Recompile: src/rebar_app_utils
Recompile: src/rebar_appups
Recompile: src/rebar_asn1_compiler
Recompile: src/rebar_base_compiler
Recompile: src/rebar_cleaner
Recompile: src/rebar_config
Recompile: src/rebar_core
Recompile: src/rebar_cover_utils
Recompile: src/rebar_ct
src/rebar_ct.erl:291:30: Warning: crypto:rand_uniform/2 is deprecated; use rand:uniform/1 instead
%  291|     Random = integer_to_list(crypto:rand_uniform(0, 10000)),
%     |                              ^

Recompile: src/rebar_deps
Recompile: src/rebar_dia_compiler
Recompile: src/rebar_dialyzer
Recompile: src/rebar_edoc
Recompile: src/rebar_erlc_compiler
Recompile: src/rebar_erlydtl_compiler
Recompile: src/rebar_escripter
Recompile: src/rebar_eunit
src/rebar_eunit.erl:282:20: Warning: crypto:rand_uniform/2 is deprecated; use rand:uniform/1 instead
%  282|             Seed = crypto:rand_uniform(1, 65535),
%     |                    ^

Recompile: src/rebar_file_utils
Recompile: src/rebar_getopt
Recompile: src/rebar_lfe_compiler
Recompile: src/rebar_log
Recompile: src/rebar_metacmds
Recompile: src/rebar_mustache
Recompile: src/rebar_neotoma_compiler
Recompile: src/rebar_otp_app
Recompile: src/rebar_otp_appup
Recompile: src/rebar_port_compiler
Recompile: src/rebar_proto_compiler
Recompile: src/rebar_proto_gpb_compiler
Recompile: src/rebar_protobuffs_compiler
Recompile: src/rebar_qc
Recompile: src/rebar_rand_compat
Recompile: src/rebar_rel_utils
Recompile: src/rebar_reltool
Recompile: src/rebar_require_vsn
Recompile: src/rebar_shell
Recompile: src/rebar_subdirs
Recompile: src/rebar_templater
Recompile: src/rebar_upgrade
Recompile: src/rebar_utils
Recompile: src/rebar_xref
Recompile: src/rmemo
==> rebar (compile)
==> rebar (escriptize)
Congratulations! You now have a self-contained script called "rebar" in
your current working directory. Place this script anywhere in your path
and you can use rebar to build OTP-compliant apps.
/app/src
make[1]: Entering directory '/app/src/deps/relx'
 DEP    bbmustache (1.10.0)
make[2]: Entering directory '/app/src/deps/bbmustache'
 DEPEND bbmustache.d
 ERLC   bbmustache.erl
 APP    bbmustache.app.src
make[2]: Leaving directory '/app/src/deps/bbmustache'
 DEPEND relx.d
 ERLC   relx.erl rlx_app_info.erl rlx_assemble.erl rlx_config.erl rlx_file_utils.erl rlx_log.erl rlx_overlay.erl rlx_release.erl rlx_relup.erl rlx_resolve.erl rlx_state.erl rlx_string.erl rlx_tar.erl rlx_util.erl
 APP    relx.app.src
make[1]: Leaving directory '/app/src/deps/relx'
 DEPEND docker_erlang_mk.d
 ERLC   docker_erlang_mk_app.erl docker_erlang_mk_sup.erl
 APP    docker_erlang_mk
erl +A1 -noinput -boot no_dot_erlang -pa ebin/ -pz /app/src/.erlang.mk/rebar/ebin -eval "   {ok, Config} = file:consult(\"/app/src/relx.config\"),  {release, {Name, Vsn0}, _} = lists:keyfind(release, 1, Config), Vsn = case Vsn0 of      {cmd, Cmd} -> os:cmd(Cmd);      semver -> \"\";     {semver, _} -> \"\";        VsnStr -> Vsn0  end,    {ok, _} = relx:build_release(#{name => Name, vsn => Vsn}, Config),  halt(0)." -- erlang.mk
Solving Release docker_erlang_mk_release-1{"init terminating in do_boot",{{error,{rlx_resolve,{app_not_found,docker_erlang_mk,undefined}}},[{rlx_resolve,subset,7,[{file,"src/rlx_resolve.erl"},{line,72}]},{rlx_resolve,'-fold_apps/7-fun-0-',7,[{file,"src/rlx_resolve.erl"},{line,99}]},{lists,foldl,3,[{file,"lists.erl"},{line,1267}]},{rlx_resolve,subset,5,[{file,"src/rlx_resolve.erl"},{line,54}]},{rlx_resolve,solve_release,2,[{file,"src/rlx_resolve.erl"},{line,48}]},{relx,build_release_,3,[{file,"src/relx.erl"},{line,170}]},{relx,build_release,3,[{file,"src/relx.erl"},{line,87}]},{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,685}]}]}}
init terminating in do_boot ({{error,{rlx_resolve,{app_not_found,docker_erlang_mk,undefined}}},[{rlx_resolve,subset,7,[{_},{_}]},{rlx_resolve,-fold_apps/7-fun-0-,7,[{_},{_}]},{lists,foldl,3,[{_},{_}]},{rlx_resolve,subset,5,[{_},{_}]},{rlx_resolve,solve_release,2,[{_},{_}]},{relx,build_release_,3,[{_},{_}]},{relx,build_release,3,[{_},{_}]},{erl_eval,do_apply,6,[{_},{_}]}]})

Crash dump is being written to: erl_crash.dump...done
make: *** [erlang.mk:7539: relx-rel] Error 1
The command '/bin/sh -c make distclean  && make' returned a non-zero code: 2

PS: writing this, I found out that if I don't bootstrap-rel, docker build . works. So the issue is only on the release part.

essen commented 2 years ago

directory '/app/src/deps/relx'

The directory of the application must be the same name as the application, so harvester, with an optional version allowed such as harvester-1.2.3. If you put your application in app the OTP release handling code won't find it.

benjamin-bergia commented 2 years ago

Good catch. I copied my code under /harvester and it works just fine.

Thank you very much.