Cross compilation issue in membrane_portaudio_plugin [macOS -> Linux]

[guillego]

Hello! I'm trying to compile (in Apple Silicon) a nerves membrane application to run on my RPi4, I was hoping membrane_portaudio_plugin:#51 would have addressed this issue but it seems like there is a lingering error in the Bundlex configuration that I can't quite figure out. The application compiles if I don't include the portaudio dependency but fails as soon as I add it.

This is the error I get:

mix deps.compile bundlex
==> nerves
==> nerves_livebook

Nerves environment
  MIX_TARGET:   rpi4
  MIX_ENV:      dev

==> bundlex
Compiling 38 files (.ex)
error: invalid quoted expression: %{os: "linux", architecture: "aarch64", vendor: "unknown", abi: "gnu"}

Please make sure your quoted expressions are made of valid AST nodes. If you would like to introduce a value into the AST, such as a four-element tuple or a map, make sure to call Macro.escape/1 before
└─ lib/bundlex.ex

== Compilation error in file lib/bundlex.ex ==
** (CompileError) lib/bundlex.ex: cannot compile file (errors have been logged)
    lib/bundlex.ex:67: (module)
could not compile dependency :bundlex, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile bundlex --force", update it with "mix deps.update bundlex" or clean it with "mix deps.clean bundlex"

System information OS: macOS Sonoma 14.4 Chip: Apple M1 Pro Erlang version: 26.2.2 Elixir version: 1.16.2-otp-26

Relevant dependencies

{:nerves_system_rpi4, "~> 1.26", runtime: false, targets: :rpi4},
{:membrane_core, "~> 1.0", targets: :rpi4_video},
{:membrane_portaudio_plugin, "~> 0.19", targets: :rpi4},
{:membrane_h264_format, "~> 0.6", targets: :rpi4},
{:membrane_h26x_plugin, "~> 0.10", targets: :rpi4},
{:membrane_mp4_format, "~> 0.8", targets: :rpi4,
{:membrane_mp4_plugin, "~> 0.33", targets: :rpi4},
{:membrane_rpicam_plugin, "~> 0.1", targets: :rpi4},
{:membrane_http_adaptive_stream_plugin, "~> 0.18", targets: :rpi4},

[mat-hek]

👋 thanks for reporting, please check if this fix helps: https://github.com/membraneframework/bundlex/tree/fix-crosscompile

[guillego]

Wow, thanks for the fast response @mat-hek! Just checked with that branch as a dependency and it works!

mix deps.compile bundlex
==> nerves
==> nerves_livebook

Nerves environment
  MIX_TARGET:   rpi4
  MIX_ENV:      dev

==> bundlex
    warning: function get_host!/0 is unused
    │
 66 │   defp get_host!() do
    │        ~
    │
    └─ lib/bundlex/platform.ex:66:8: Bundlex.Platform (module)

I now face some additional issues with portaudio compilation though but perhaps that should go in another issue?

==> membrane_portaudio_plugin
Bundlex: Building natives: sink, source, pa_devices
warning: Bundlex: couldn't load ["portaudio-2.0"] libraries with pkg-config due to:
    ** (BundlexError) pkg-config error:
    Code: 1
    Package portaudio-2.0 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `portaudio-2.0.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'portaudio-2.0' found

could not compile dependency :membrane_portaudio_plugin, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile membrane_portaudio_plugin --force", update it with "mix deps.update membrane_portaudio_plugin" or clean it with "mix deps.clean membrane_portaudio_plugin"
==> nerves_livebook
** (Mix) Bundlex: Couldn't load OS dependency :portaudio of package membrane_portaudio_plugin. Make sure to follow installation instructions that may be available in the readme of membrane_portaudio_plugin.

Tried the following providers:

Provider `{:precompiled, nil}` ignored, no URL provided
Provider `{:pkg_config, "portaudio-2.0"}` couldn't load ["portaudio-2.0"] libraries with pkg-config due to:
    ** (BundlexError) pkg-config error:
    Code: 1
    Package portaudio-2.0 was not found in the pkg-config search path.
    Perhaps you should add the directory containing `portaudio-2.0.pc'
    to the PKG_CONFIG_PATH environment variable
    No package 'portaudio-2.0' found

[guillego]

I pointed the PKG_CONFIG_PATH to my homebrew portaudio package:

export PKG_CONFIG_PATH="/opt/homebrew/Cellar/portaudio/19.7.0/lib/pkgconfig"

However it now fails due to not finding portaudio.h, which is in the /include directory as specified in the pkgconfig/portaudio-2.0.pc file.

==> membrane_portaudio_plugin
Bundlex: Building natives: sink, source, pa_devices
In file included from /Users/guille/repos/3party/nerves_livebook/deps/membrane_portaudio_plugin/c_src/membrane_portaudio_plugin/_generated/nif/sink.h:8,
                 from /Users/guille/repos/3party/nerves_livebook/deps/membrane_portaudio_plugin/c_src/membrane_portaudio_plugin/_generated/nif/sink.c:1:
/Users/guille/repos/3party/nerves_livebook/deps/membrane_portaudio_plugin/c_src/membrane_portaudio_plugin/_generated/nif/../../sink.h:8:10: fatal error: portaudio.h: No such file or directory
    8 | #include <portaudio.h>
      |          ^~~~~~~~~~~~~
compilation terminated.
could not compile dependency :membrane_portaudio_plugin, "mix compile" failed."

[Noarkhh]

Hi!

When cross-compiling using Nerves the headers need to be available in the target Nerves System, not on the host device. The reason for that is the fact that the libraries files that are available locally are compiled for your host's machine architecture, not for the one of the Nerves target.

The headers that are available during cross-compilation can be found in ~/.nerves/artifacts/<nerves_system_name>/staging/usr/include. If portaudio.h is not present there, then it won't be available during cross-compilation. As far as I know the most correct way to include additional dependencies in your Nerves System is to create a custom one, which is explained here.

[guillego]

Perfect! I'll add portaudio to my custom system then, thanks for all the help @Noarkhh and @mat-hek!

[guillego]

portaudio is now available on my custom nerves system and that got the compilation step through! Thanks @Noarkhh

Seems like I still have one more hurdle to go related to the cross-compilation. It seems like Bundlex can't load the compiled sources in priv/.

mix compile
==> nerves
==> nerves_system_rpi4_video
Generated nerves_system_rpi4_video app
==> nerves_livebook

Nerves environment
  MIX_TARGET:   rpi4_video
  MIX_ENV:      dev

==> membrane_portaudio_plugin
Bundlex: Building natives: sink, source, pa_devices
Compiling 8 files (.ex)

15:37:30.897 [error] Bundlex cannot load nif :pa_devices of app :membrane_portaudio_plugin
from "/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/pa_devices", check bundlex.exs file for information about nifs.
Reason: :load_failed, Failed to load NIF library: 'dlopen(/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/pa_devices.so, 0x0002): tried: '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/pa_devices.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/pa_devices.so' (no such file), '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/pa_devices.so' (not a mach-o file)'

15:37:30.898 [error] Bundlex cannot load nif :sink of app :membrane_portaudio_plugin
from "/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/sink", check bundlex.exs file for information about nifs.
Reason: :load_failed, Failed to load NIF library: 'dlopen(/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/sink.so, 0x0002): tried: '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/sink.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/sink.so' (no such file), '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/sink.so' (not a mach-o file)'

15:37:30.898 [error] Bundlex cannot load nif :source of app :membrane_portaudio_plugin
from "/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/source", check bundlex.exs file for information about nifs.
Reason: :load_failed, Failed to load NIF library: 'dlopen(/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/source.so, 0x0002): tried: '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/source.so' (not a mach-o file), '/System/Volumes/Preboot/Cryptexes/OS/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/source.so' (no such file), '/Users/guille/repos/3party/nerves_livebook/_build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif/source.so' (not a mach-o file)'

Generated membrane_portaudio_plugin app
==> nerves_livebook
Generated nerves_livebook app

$ ls _build/rpi4_video_dev/lib/membrane_portaudio_plugin/priv/bundlex/nif
pa_devices.so  pa_devices_obj sink.so        sink_obj       source.so      source_obj

[Noarkhh]

It's expected, bundlex tries to load NIFs that are compiled for different architecture than current one (mach-o file is a macos format). This shouldn't be a problem, since Bundlex will try to load them on the target platform also, whose architecture is compatible with the NIFs.

On the actual platform they should be loaded just fine.

[guillego]

Thanks @Noarkhh! I tried flashing a Nerves firmware with these compiled dependencies but I can't get it to run. If I compile the same firmware without the bundlex and membrane_portaudio_plugin dependencies it runs without issue, but as soon as I flash the firmware that contains those deps it doesn't boot.

Not sure if at this point this is a Nerves issue or a membrane issue. Let me know if there's anything I can do to help understand the issue better.

edit: these are the deps for bundlex and membrane_portaudio_plugin that I'm using

      {:bundlex, github: "membraneframework/bundlex",
      branch: "fix-crosscompile",
      targets: :rpi4_video,
      override: true},
      {:membrane_portaudio_plugin, "~> 0.19", targets: :rpi4_video},

[Noarkhh]

Hm, that's a bit concerning. How do you connect to your target?

[guillego]

Either via WiFi or the USB-C connector (OTG). At this point I'm only testing some video streaming pipelines so I'm using https://github.com/nerves-livebook/nerves_livebook with the extra membrane dependencies.

Basically I compile and burn the firmware and then I can ssh into it (or access livebook on the browser) when I connect it to my laptop with the OTG cable. Then I configure the WiFi.

[Noarkhh]

Would you be able to connect to the Raspberry Pi directly via HDMI and keyboard? Maybe that could be useful in diagnosing the problem.

[guillego]

I'll need to get a mini HDMI cable, I could get it next week and then check what's going on at boot.

I also re-flashed the firmware that was not working with the --upgrade flag in fwup to not override /root in hopes that I could see the erlang crash.dump file but there was nothing there.

[Noarkhh]

I managed to recreate your issue and it seems to go deeper. Even after attempts to stop the RPi from rebooting after BEAM crashes (as described here) it's still rebooting. No information is being printed on my monitor either.

I'll keep looking into it, if you find any clues what might be the cause of this feel free to share them :)

[guillego]

Thanks a lot @Noarkhh! I just got a mini HDMI cable so I'll be looking into it this week. It does look like a deep rabbit hole though 😅

[CJRChang]

Stumbled across this issue and in case it helps - I've been running membrane_portaudio_plugin (for microphone input in my case) since 0.16.0 (early-mid 2023?) and have had it working through to ~0.18.3 (my current fork here which wasn't upstreamed since another solution was preferable) with cross compilation on Apple silicon (M1) hardware to rpi4 (using bundlex 1.4.5 specifically).

I haven't tried since ~0.18.3 due to time constraints (and issues described in this MR but perhaps that might work for you too.

[Noarkhh]

Hi!

Thanks for your insight @CJRChang, providing the bundlex version saved me a lot of time when debugging :)

I think I magaged to fix the bug @guillego, you can try this branch: https://github.com/membraneframework/bundlex/tree/fix-nerves-crashes

[Noarkhh]

Hi @guillego, I released the fix in bundlex 1.5.1, you can give it a try :)

[darthez]

Hi @guillego, did you get a chance to try the 1.5.1 version?

[guillego]

Hey there @Noarkhh and @darthez! Thanks a lot for the work in the new Bundlex version.

Sorry I haven't replied yet, I'm traveling these days and don't have access to a RPi 4 right now. Will not be able to test it until mid June. If you want we can consider this closed and I'll reopen if the issue arises again once I can test it!

[Noarkhh]

All right, feel free to reopen should any problems related to this issue arise :)