Error in compiling FastPyRateC library

[LauravanHolstein]

Hello!

I am trying to compile and install the FastPyRateC library on a Mac (M1 chip). At the "compile the library" step, I get the following errors:

`FastPyRateC.cpp:8:10: fatal error: 'boost/math/distributions.hpp' file not found

include <boost/math/distributions.hpp>

     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 error generated. error: command '/usr/bin/clang' failed with exit code 1 Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ python3 setup.py build FastPyRateC.cpp:8:10: fatal error: 'boost/math/distributions.hpp' file not found

include <boost/math/distributions.hpp>

     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

1 error generated. error: command '/usr/bin/clang' failed with exit code 1 Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ cd /Users/lauravanholstein/PyRate/PyRate-master/pyrate_lib/fastPyRateC/ModulePyrateC Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ mv boost_1_77_0/boost usage: mv [-f | -i | -n] [-hv] source target mv [-f | -i | -n] [-v] source ... directory Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ rm -r boost_1_77_0 Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ swig -c++ -python FastPyRateC.i Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ xcode-select --install xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ python setup.py build -bash: python: command not found Lauras-MacBook-Pro-6:ModulePyrateC lauravanholstein$ python3 setup.py build In file included from FastPyRateC.cpp:8: In file included from ./boost/math/distributions.hpp:15: In file included from ./boost/math/distributions/arcsine.hpp:35: In file included from ./boost/math/constants/constants.hpp:14: In file included from ./boost/math/tools/convert_from_string.hpp:14: In file included from ./boost/math/tools/lexical_cast.hpp:12: In file included from ./boost/lexical_cast.hpp:32: In file included from ./boost/lexical_cast/try_lexical_convert.hpp:44: In file included from ./boost/lexical_cast/detail/converter_lexical.hpp:54: ./boost/lexical_cast/detail/converter_lexical_streams.hpp:285:21: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(begin, ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

In file included from FastPyRateC.cpp:8: In file included from ./boost/math/distributions.hpp:15: In file included from ./boost/math/distributions/arcsine.hpp:35: In file included from ./boost/math/constants/constants.hpp:14: In file included from ./boost/math/tools/convert_from_string.hpp:14: In file included from ./boost/math/tools/lexical_cast.hpp:12: In file included from ./boost/lexical_cast.hpp:32: In file included from ./boost/lexical_cast/try_lexical_convert.hpp:44: In file included from ./boost/lexical_cast/detail/converter_lexical.hpp:54: ./boost/lexical_cast/detail/converter_lexical_streams.hpp:297:21: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(begin, ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

In file included from FastPyRateC.cpp:8: In file included from ./boost/math/distributions.hpp:15: In file included from ./boost/math/distributions/arcsine.hpp:35: In file included from ./boost/math/constants/constants.hpp:14: In file included from ./boost/math/tools/convert_from_string.hpp:14: In file included from ./boost/math/tools/lexical_cast.hpp:12: In file included from ./boost/lexical_cast.hpp:32: In file included from ./boost/lexical_cast/try_lexical_convert.hpp:44: In file included from ./boost/lexical_cast/detail/converter_lexical.hpp:54: ./boost/lexical_cast/detail/converter_lexical_streams.hpp:310:21: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(begin, ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

3 warnings generated. FastPyRateC_wrap.cxx:4177:11: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(msg, "attempt to assign sequence of size %lu to extended slice of size %lu", (unsigned long)is.size(), (unsigned long)replacecount); ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

FastPyRateC_wrap.cxx:4193:9: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(msg, "attempt to assign sequence of size %lu to extended slice of size %lu", (unsigned long)is.size(), (unsigned long)replacecount); ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

FastPyRateC_wrap.cxx:4533:2: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(msg, "in sequence element %d ", (int)_index); ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

3 warnings generated. `

Any help would be appreciated! Many thanks, Laura

[LauravanHolstein]

Apparently in macOS 13, sprintf was deprecated in favour of vsprintf. This helped: https://developer.apple.com/forums/thread/714675

[thauffe]

Hi Laura,

Could you please post here your steps to compile the FastPyRateC library? Where exactly did you replace sprintf with vsprintf?

More users seem to have the same issue on a M1 Macs and we would like to update the tutorial with your fix.

Thanks a lot!

[LauravanHolstein]

Hi Torsten!

Turns out it's not a fix at all... Just tried to summarise model probabilities (because for some reason I was able, after my "fix", to run an RJMCMC), and I got the Module FastPyRateC not found error. I checked and the same message was generated during the RJMCMC, but it ran anyway (?), and I didn't see it.

Hmmm...not sure what to do. Any ideas? Happy to try things out on my machine. -Laura

[thauffe]

Hi Laura,

RJMCMC and PyRate in general works without the FastPyRateC library and gives the same result. It's just slower.

We tried to compile the library manually on an M1 machine and could solve your initial error. The boost library, on which the library depends, was not downloaded correctly. However, eventually we got stuck with a different error related to where M1 Macs expect a compiler to be versus the place on older Macs. Maybe you can try if what happens on yours?

1. Navigate to the correct folder cd .../pyrate_lib/fastPyRateC/ModulePyrateC

2. Download boost, unpack, rename, and clean curl https://boostorg.jfrog.io/artifactory/main/release/1.82.0/source/boost_1_82_0.zip -L -o boost_1_82_0.zip This is a single command unzip -q boost_1_82_0.zip mv boost_1_82_0/boost . rm boost_1_82_0.zip rm -r boost_1_82_0

3. Create the C++/Python interface swig -c++ -python FastPyRateC.i

4. Compile the library python setup.py build

This last step starts but at one moment it requires the llvm compiler. We installed it with brew install llvm. However, llvm is not found during the compilation because it is not in /usr/local as it was the case on pre-M1 Macs. There is some discussion on how to set a path to the directory of llvm on M1 but did not get it. Maybe you have more success?

Best, Torsten

[LauravanHolstein]

Hi Torsten!

Thanks for this; the correct boost file did download now.

Hmm - I'm still getting the sprintf deprecated error:

This is with the new boost (1.82.0). I'm running this on Python 3.11 (so I run python3 setup.py build) - but I assume this makes no difference.. (?)

After this error, in the 'build' folder, I get a file called lib.macosx-13-arm64-cpython-311, with a file called "_FastPyRateC.cpython-311-darwin.so". No "_FastPyRateC.so" anywhere. Happy to send the "_FastPyRateC.cpython-311-darwin.so" file if useful.

I'll keep trying to work around the sprintf error & will keep you posted.

• L

[thauffe]

Hi Laura,

You did it! The sprintf is only causing warnings but no error and the "_FastPyRateC.cpython-311-darwin.so" file is what you need. Now, you just need to rename it into "_FastPyRateC.so" and move it to the pyrate_lib/fastPyRateC/macOS folder.

The name of the compiled file seems to be different then at the time of writing the tutorial. This is not M1 specific. For instance, on my computer it is called "_FastPyRateC.cpython-311-x86_64-linux-gnu.so"

Typing python3 setup.py build is more explicit than just python and only matters if you have python v2 on your machine.

Please check that when you run PyRate it reports Module FastPyRateC was loaded before showing the first MCMC iteration.

Would be great if you could send us your compiled _FastPyRateC.so file to see if it works to just plug it into other M1 Macs.

Thanks a lot!

[LauravanHolstein]

Super! Thanks so much for your help. Here it is: _FastPyRateC.so.zip

Best wishes, Laura

[thauffe]

Hi Laura,

Thanks! Was the FastPyRate module loaded on your machine? Could you pls verify it by checking the message right at the start of PyRate?

[LauravanHolstein]

Yes, it loaded! Woohoo!

[thauffe]

This is awesome! Maybe you are the first one using this module on this new Macs?

Did you use a native python installation that was specifically build for M1 or python translated for arm chips with Rosetta?

You can check this by going to your terminal:

1. Start python python
2. import a library import platform
3. check what type of processor is recognized (arm for M1 or i386 for x86 cpus) platform.processor()
4. exit python quit()

Edit: We could use your compiled FastPyRateC on another modern Mac with a python version build specifically for M. Other users won't need to compile it themself because from now on your version should be loaded automatically. Thanks again!

[LauravanHolstein]

Hi Torsten,

My apologies for the late reply - I was on holiday! Just checked, and (in case it's still useful) it's arm for M1.

That's fab! Happy it all worked out well!

Best wishes, Laura

[thauffe]

Hi Laura,

Hope you had a good time and thanks for checking your python build!

Best, Torsten