search

metalbear-co / mirrord

Connect your local process and your cloud environment, and run local code in cloud conditions.
https://mirrord.dev
MIT License
3.59k stars 100 forks source link

Elixir Issues with Compiled C NIF Library #2492

Open bradschwartz opened 1 month ago

bradschwartz commented 1 month ago

Bug Description

A Bcrypt hashing library for Elixir seems to have troubled when it is compiled by a process that is being run via mirrord. It is implemented as a NIF (FFI) and requires a C compiler to build. It seems like when I build my dependencies locally outside of mirrord, things work properly, but if I use mirrord without having pre-compiled the library is not compiling successfully (but fails silently until runtime).

Steps to Reproduce

  1. You can clone my repro demo app: https://github.com/bradschwartz/mirrord_demo
  2. Download dependencies: mix deps.get
  3. Make sure to remove any compilation artifacts: rm -rf _build
  4. Run the elixir shell via mirrord, which should do the compilation now: mirrord exec -- iex -S mix
  5. In the iex shell, run: Bcrypt.Base.gen_salt(4, false)

Backtrace

09:23:00.543 [warning] The on_load function for module Elixir.Bcrypt.Base returned:
{%RuntimeError{
   message: "An error occurred when loading Bcrypt.\nMake sure you have a C compiler and Erlang 20 installed.\nIf you are not using Erlang 20, either upgrade to Erlang 20 or\nuse version 0.12 of bcrypt_elixir.\nSee the Comeonin wiki for more information.\n"
 },
 [
   {Bcrypt.Base, :init, 0,
    [file: 'lib/bcrypt/base.ex', line: 15, error_info: %{...}]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: 'code_server.erl', line: 1317]}
 ]}

09:23:00.542 [error] Process #PID<0.1389.0> raised an exception
** (RuntimeError) An error occurred when loading Bcrypt.
Make sure you have a C compiler and Erlang 20 installed.
If you are not using Erlang 20, either upgrade to Erlang 20 or
use version 0.12 of bcrypt_elixir.
See the Comeonin wiki for more information.

    (bcrypt_elixir 3.1.0) lib/bcrypt/base.ex:15: Bcrypt.Base.init/0
    (kernel 8.5.4) code_server.erl:1317: anonymous fn/1 in :code_server.handle_on_load/5

09:23:00.547 [error] Process #PID<0.1391.0> raised an exception
** (RuntimeError) An error occurred when loading Bcrypt.
Make sure you have a C compiler and Erlang 20 installed.
If you are not using Erlang 20, either upgrade to Erlang 20 or
use version 0.12 of bcrypt_elixir.
See the Comeonin wiki for more information.

    (bcrypt_elixir 3.1.0) lib/bcrypt/base.ex:15: Bcrypt.Base.init/0
    (kernel 8.5.4) code_server.erl:1317: anonymous fn/1 in :code_server.handle_on_load/5

09:23:00.547 [warning] The on_load function for module Elixir.Bcrypt.Base returned:
{%RuntimeError{
   message: "An error occurred when loading Bcrypt.\nMake sure you have a C compiler and Erlang 20 installed.\nIf you are not using Erlang 20, either upgrade to Erlang 20 or\nuse version 0.12 of bcrypt_elixir.\nSee the Comeonin wiki for more information.\n"
 },
 [
   {Bcrypt.Base, :init, 0,
    [file: 'lib/bcrypt/base.ex', line: 15, error_info: %{...}]},
   {:code_server, :"-handle_on_load/5-fun-0-", 1,
    [file: 'code_server.erl', line: 1317]}
 ]}

** (UndefinedFunctionError) function Bcrypt.Base.gen_salt/2 is undefined (module Bcrypt.Base is not available)
    (bcrypt_elixir 3.1.0) Bcrypt.Base.gen_salt(4, false)

Relevant Logs

Relevant logs from compilation process:

==> bcrypt_elixir
mkdir -p "/Users/brad.schwartz/code/bradschwartz/mirrord_demo/_build/dev/lib/bcrypt_elixir/priv"
cc -g -O3 -Wall -Wno-format-truncation -I"/Users/brad.schwartz/.asdf/installs/erlang/25.3.2.2/erts-13.2.2.1/include" -Ic_src -fPIC -shared -dynamiclib -undefined dynamic_lookup c_src/bcrypt_nif.c c_src/blowfish.c -o "/Users/brad.schwartz/code/bradschwartz/mirrord_demo/_build/dev/lib/bcrypt_elixir/priv/bcrypt_nif.so"
warning: unknown warning option '-Wno-format-truncation' [-Wunknown-warning-option]
1 warning generated.
warning: unknown warning option '-Wno-format-truncation' [-Wunknown-warning-option]
1 warning generated.
Compiling 3 files (.ex)
Generated bcrypt_elixir app

Your operating system and version

Darwin Brad-Schwartz.lan 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000 arm64

Local process

/Users/brad.schwartz/.asdf/shims/iex: Bourne-Again shell script text executable, ASCII text

Local process version

Erlang/OTP 25 [erts-13.2.2.1] [source] [64-bit] [smp:10:10] [ds:10:10:10] [async-threads:1] IEx 1.14.5 (compiled with Erlang/OTP 24)

Additional Info

I've tried setting skip_build_tools: false and that didn't do anything.

bradschwartz commented 1 month ago

Summarizing from discussion over Discord:

mirrord is built as a universal binary for Mac devices. This is why when you run a command to print the architecture, it will give results that look like it's running either as 32-bit or 64-bit x86 machine:

$ mirrord exec -- uname -p
i386
$ mirrord exec -- uname -m
x86_64

What's most likely happening is that bcrypt is then building an x86 shared object:

$ file _build/dev/lib/bcrypt_elixir/priv/bcrypt_nif.so
_build/dev/lib/bcrypt_elixir/priv/bcrypt_nif.so: Mach-O 64-bit dynamically linked shared library x86_64

and mix is having issues loading that properly once it tries to run any relevant commands.

The best workaround is to precompile dependencies mix deps.compile in advance of starting up the process via mirrord.