Odd issue with MDX when running in Docker

[mt-caret]

When trying to build the project in a dev container, we seem to get the following linking-related error:

# dune build @fmt @runtest @doc  
File "lib/dune", line 41, characters 0-123:
41 | (mdx
42 |  (files :standard - *.mli)
43 |  (libraries core core_kernel.nonempty_list polars)
44 |  (preludes mdx_prelude/mdx_prelude.txt))
Fatal error: cannot load shared library dllpolars_ocaml
Reason: dllpolars_ocaml.so: cannot open shared object file: No such file or directory

This happens on both Docker for mac and on GitHub codespaces. I suspect we're hitting some issue related to how OCaml links things into bytecode binaries, but I'm not entirely sure. Commenting out the mdx stanza fixes the issue.

Passing some options, we get:

# dune runtest --verbose -j1
Running[19]: (cd _build/default/lib && ./mdx_gen.bc utils.mli) > _build/default/lib/.mdx/utils.mli.corrected
File "lib/dune", line 41, characters 0-123:          
41 | (mdx
42 |  (files :standard - *.mli)
43 |  (libraries core core_kernel.nonempty_list polars)
44 |  (preludes mdx_prelude/mdx_prelude.txt))
Command [19] got signal ABRT:
$ (cd _build/default/lib && ./mdx_gen.bc utils.mli) > _build/default/lib/.mdx/utils.mli.corrected
Fatal error: cannot load shared library dllpolars_ocaml
Reason: dllpolars_ocaml.so: cannot open shared object file: No such file or directory

Running mdx_gen.bc on its own also seems to exhibit the issue:

_build/default/lib/mdx_gen.bc 
Fatal error: cannot load shared library dllpolars_ocaml
Reason: dllpolars_ocaml.so: cannot open shared object file: No such file or directory
Aborted (core dumped)

[mt-caret]

Ok, so it seems like the LD_LIBRARY_PATH is not being set correctly here.

# (cd _build/default/lib; CAML_LD_LIBRARY_PATH=$CAML_LD_LIBRARY_PATH:./ ./mdx_gen.bc utils.mli)
open Core

val string_result_ok_exn : ('a, string) result -> 'a

works fine, but it's unclear how to ask dune to do this correctly. Apparently there is an open issue tracking this here: https://github.com/ocaml/dune/issues/108#issuecomment-1841585863 I'm not sure why it was working before, though; perhaps this is a regression in dune?

[mt-caret]

I tried using dune 3.8.0 (instead of 3.11.1 which is what I got in the dev container) and I got sendFile-related issues instead. I remember seeing these a while back. Trying out 3.8.3 next which mentions some fixes related to sendFile to see if that gets something working.

[mt-caret]

I tried a few different versions and all of them either exhibit the sendFile issue or fail to find the .so file.