clang argument error during model compile on MacOS

[fonnesbeck]

I'm currently not able to compile stan models on MacOS (Ventura) through bridgestan (run via nutpie). clang complains that it does not recognize the argument '-flifetime-dse=1'

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[5], line 1
----> 1 compiled_model = nutpie.compile_stan_model(code=model_string)

File [~/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py:149](https://file+.vscode-resource.vscode-cdn.net/Users/cfonnesbeck/phillies/pie/~/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py:149), in compile_stan_model(code, filename, extra_compile_args, extra_stanc_args, dims, coords, model_name, cleanup)
    [147](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=146) if extra_stanc_args:
    [148](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=147)     stanc_args.extend(extra_stanc_args)
--> [149](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=148) so_path = bridgestan.compile_model(
    [150](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=149)     model_path, make_args=make_args, stanc_args=stanc_args
    [151](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=150) )
    [152](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=151) # Set necessary library loading paths
    [153](file:///Users/cfonnesbeck/GitHub/nutpie/src/nutpie/python/nutpie/compile_stan.py?line=152) bridgestan.compile.windows_dll_path_setup()

File [~/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py:129](https://file+.vscode-resource.vscode-cdn.net/Users/cfonnesbeck/phillies/pie/~/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py:129), in compile_model(stan_file, stanc_args, make_args)
    [123](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=122) if proc.returncode:
    [124](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=123)     error = (
    [125](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=124)         f"Command {' '.join(cmd)} failed with code {proc.returncode}.\n"
    [126](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=125)         f"stdout:\n{proc.stdout}\nstderr:\n{proc.stderr}"
    [127](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=126)     )
--> [129](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=128)     raise RuntimeError(error)
    [130](file:///Users/cfonnesbeck/mambaforge/envs/pymc/lib/python3.11/site-packages/bridgestan/compile.py?line=129) return output

RuntimeError: Command make STAN_THREADS=true STANCFLAGS=--include-paths=. [/private/var/folders/40/_ljkntgn08g0vn2_s9gbxs340000gn/T/tmp2h_i3ptf/model_model.so](https://file+.vscode-resource.vscode-cdn.net/private/var/folders/40/_ljkntgn08g0vn2_s9gbxs340000gn/T/tmp2h_i3ptf/model_model.so) failed with code 2.
stdout:
tbb_root="../tbb_2020.3" CXX="arm64-apple-darwin20.0.0-clang++" CC="arm64-apple-darwin20.0.0-clang" LDFLAGS='-Wl,-L,"/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb"  ' 'make' -C "./stan/lib/stan_math/lib/tbb" -r -f "/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb_2020.3/build/Makefile.tbbmalloc" compiler=gcc cfg=release stdver=c++1y malloc CXXFLAGS="-Wno-unknown-warning-option -Wno-deprecated-copy  "
make[1]: Entering directory '/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb'
arm64-apple-darwin20.0.0-clang++ -c -MMD -O2 -DUSE_PTHREAD  -isysroot [/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk](https://file+.vscode-resource.vscode-cdn.net/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk)   -Wno-unknown-warning-option -Wno-deprecated-copy   -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -fno-rtti -fno-exceptions -D__TBBMALLOC_BUILD=1 -Wno-non-virtual-dtor -fPIC -flifetime-dse=1 -I../tbb_2020.3/src -I../tbb_2020.3/src/rml/include -I../tbb_2020.3/include -I../tbb_2020.3/src/tbbmalloc -I../tbb_2020.3/src/tbbmalloc ../tbb_2020.3/src/tbbmalloc/backend.cpp
make[1]: Leaving directory '/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb'

stderr:
clang-15: error: unknown argument: '-flifetime-dse=1'
make[1]: *** [/Users/cfonnesbeck/.bridgestan/bridgestan-2.1.2/stan/lib/stan_math/lib/tbb_2020.3/build/Makefile.tbbmalloc:65: backend.o] Error 1
make: *** [stan/lib/stan_math/make/libraries:173: stan/lib/stan_math/lib/tbb/tbbmalloc.def] Error 2

Running bridgestan 2.1.2 and Python 3.11.0.

[WardBrian]

Hi @fonnesbeck -

This is a strange error, since it comes from one of the Stan-math dependencies, but as far as I can tell they only pass that argument when using GCC, not clang: https://github.com/search?q=repo%3Astan-dev%2Fmath+lifetime-dse+NOT+language%3AText&type=code

Do you use any other Stan interface (e.g. CmdStan)? Was this error present there?

[fonnesbeck]

Yes, I just ran the model successfully using the same data under cmdstanpy. Not sure if their default compile args are the same as under bridgestan or not.

[WardBrian]

Could you show the output of make -d?

there shouldn’t be any differences between cmdstan’s tbb make rules and bridgestans, but clearly something is up

[fonnesbeck]

I've been running bridgestan via nutpie. Do you just want make -d from within the bridgestan source directory?

[WardBrian]

Running make manually would definitely work, but you might be able to pass -v as an extra argument in the nutpie wrapper

[WardBrian]

@fonnesbeck thanks - could you try the same thing but building a model? for example make -v test_models/multi/multi_model.so from the bridgestan folder

[fonnesbeck]

The -v flag just returns the make version. Running with -d seems to hang here when running the multi_model example:

Reading makefiles...
Reading makefile '/Users/cfonnesbeck/GitHub/bridgestan/stan/lib/stan_math/lib/tbb_2020.3/build/Makefile.tbbmalloc'...
Reading makefile '../tbb_2020.3/build/common.inc' (search path) (no ~ expansion)...
make[1]: Entering directory '/Users/cfonnesbeck/GitHub/bridgestan/stan/lib/stan_math/lib/tbb'
Reading makefile '../tbb_2020.3/build/macos.inc' (search path) (no ~ expansion)...
Reading makefile '../tbb_2020.3/build/macos.clang.inc' (search path) (no ~ expansion)...

[WardBrian]

My bad, I meant -d.

That's very interesting, since that file it is reading does not have the -flifetime-dse=1 argument, macos.gcc.inc does.

How long did you let it spin before killing it?

[fonnesbeck]

Still running!

[WardBrian]

Hm, very odd. If you have a working cmdstan you should be able to literally copy the tbb files from that over and it should work, but that doesn't answer why this is happening still

[fonnesbeck]

Yes, very curious. Moreover, it happens on two different machines.

[fonnesbeck]

This appears to be some odd side-effect of running Python from within VSCode. If I run the model from a terminal, the error does not occur. Not sure why, but clearly has nothing to do with bridgestan, so I will close this. Apologies!

[WardBrian]

Oh very odd! I wonder if there are any environment variables being set by VSCode which Make is then picking up

[aseyboldt]

I guess you use conda, right? Maybe it could also have something to do with environment variables set by conda? Especially if you are not using bash interactively, some variables might then not get set, even if you activate an env? I think CVCode always uses the bash activate scripts.