Closed arkorobotics closed 3 months ago
This issue appears to be related to #108, but the thread concludes the issue was fixed.
@biergaizi @aWZHY0yQH81uOYvH - Any chance this is similar or identical issue?
@arkorobotics I believe this is a different problem. Could you provide your full ~/Library/Logs/Homebrew/openems/02.make
logs? Because it's a multithreaded build, the last lines of the logs shown don't actually catch any errors.
I ran into macOS build problems recently and a PR that should fix it is waiting here: https://github.com/thliebig/openEMS/pull/137
Sure thing! Here's the ~/Library/Logs/Homebrew/openems/02.make
log file from the failing brew install:
02.make.txt
As an aside, I tried pulling the latest master for OpenEMS-Project and building from source using the following steps:
git clone --recurse-submodules -j8 https://github.com/thliebig/openEMS-Project.git
cd openEMS-Project
I added the line from the PR you mentioned to openEMS/nf2ff/CMakeLists.txt
+set_target_properties(nf2ff PROPERTIES CXX_STANDARD 11)
...but then got an HDF5 dependency error asking for version 1.8, which appears to be deprecated, so I changed HDF5 requirements in openEMS/CMakeLists.txt
and CSXCAD/CMakeLists.txt
to use HDF5 version 1.14.3 (appears to be the latest?).
-find_package(HDF5 1.8 COMPONENTS C HL REQUIRED)
+find_package(HDF5 1.14.3 COMPONENTS C HL REQUIRED)
Doing so seems to have fixed the HDF5 error, but now I'm gettin new error about QT6
-- Found OpenGL: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.4.sdk/System/Library/Frameworks/OpenGL.framework found components: OpenGL
CMake Error at /usr/local/lib/cmake/vtk-9.3/VTK-vtk-module-find-packages.cmake:1290 (find_package):
By not providing "FindQt6.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "Qt6", but
CMake did not find one.
Could not find a package configuration file provided by "Qt6" (requested
version 6.6) with any of the following names:
Qt6Config.cmake
qt6-config.cmake
Add the installation prefix of "Qt6" to CMAKE_PREFIX_PATH or set "Qt6_DIR"
to a directory containing one of the above files. If "Qt6" provides a
separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
/usr/local/lib/cmake/vtk-9.3/vtk-config.cmake:159 (include)
CMakeLists.txt:106 (find_package)
-- Configuring incomplete, errors occurred!
make[2]: *** [QCSXCAD-prefix/src/QCSXCAD-stamp/QCSXCAD-configure] Error 1
make[1]: *** [CMakeFiles/QCSXCAD.dir/all] Error 2
make: *** [all] Error 2
Not sure if this is useful or if you have seen this before, but I'm trying a different build/install avenue in the meantime.
Your log file does contain some issues related to the Boost thread library, which is what my PR should fix. (For some reason the default C++ stdlib version clang is compiling with is very old??)
There are also some issues related to openEMS itself, but hopefully that's just a result of the previous errors...
/tmp/openems-20240309-62024-ywke8h/openEMS/FDTD/operator.cpp:1716:23: error: no type named 'LEtype' in 'CSPropLumpedElement'
CSPropLumpedElement::LEtype lumpedType = p_prop->GetLEtype();
~~~~~~~~~~~~~~~~~~~~~^
/tmp/openems-20240309-62024-ywke8h/openEMS/FDTD/operator.cpp:1716:51: error: no member named 'GetLEtype' in 'CSPropLumpedElement'
CSPropLumpedElement::LEtype lumpedType = p_prop->GetLEtype();
~~~~~~ ^
/tmp/openems-20240309-62024-ywke8h/openEMS/FDTD/operator.cpp:1720:58: error: no member named 'PARALLEL' in 'CSPropLumpedElement'
bool IsParallelRC = (lumpedType == CSPropLumpedElement::PARALLEL) && !(L > 0.0);
~~~~~~~~~~~~~~~~~~~~~^
Homebrew adds lots of magic arguments to the CMake line, so it may fix those other errors you ran into when compiling manually. (e.g. Qt6 is a dependency in Homebrew, but may not be symlinked to a place where CMake can find it by default)
Ah good to know! Hopefully the PR resolves these issues.
Also, I got the manual build to work after making two more modifications:
1) Installed QT6 and linked it using Homebrew:
$ brew install qt
$ brew link qt
2) Updated AppCSXCAD/CMakeLists.txt
to use HDF5 version 1.14.3
-find_package(HDF5 1.8 COMPONENTS C HL REQUIRED)
+find_package(HDF5 1.14.3 COMPONENTS C HL REQUIRED)
I'm not sure if it's my machine's configuration or not, but do you think it's worth adding qt6 to the Mac Install Requirements Wiki Page and updating all CMakeLists.txt
files with HDF5 dependencies to use v1.14.3?
Yeah, that wiki page should probably be updated. It's a dependency for the UI elements, not the core openEMS program itself (but this repo builds everything). The dependencies Homebrew is using are listed here.
Also not really sure why there's an explicit version of HDF5 specified in the CMakeLists.txt
if it works with the newest...
For those who'd like to build openEMS-Project for macOS 14.1.2 as of 3/9/2024, here's what worked for me:
Follow macOS build instructions: https://docs.openems.de/install/requirements.html#macos
Install QT6
$ brew install qt
$ brew link qt
Clone latest master from https://github.com/thliebig/openEMS-Project - hash: d3c31d2
$ git clone --recurse-submodules -j8 https://github.com/thliebig/openEMS-Project.git
$ cd openEMS-Project
Modify the following CMakeLists.txt
files to use HDF5 v1.14.3: openEMS/CMakeLists.txt
, AppCSXCAD/CMakeLists.txt
, and CSXCAD/CMakeLists.txt
-find_package(HDF5 1.8 COMPONENTS C HL REQUIRED)
+find_package(HDF5 1.14.3 COMPONENTS C HL REQUIRED)
Added CXX_STANDARD 11 to openEMS/nf2ff/CMakeLists.txt
+set_target_properties(nf2ff PROPERTIES CXX_STANDARD 11)
Build / Install
./update_openEMS.sh ~/opt/openEMS --with-hyp2mat --with-CTB --python
... and any remaining install instructions here: https://docs.openems.de/install/clone-build-install.html#build-and-install
Yeah, that wiki page should probably be updated. It's a dependency for the UI elements, not the core openEMS program itself (but this repo builds everything). The dependencies Homebrew is using are listed here.
Also not really sure why there's an explicit version of HDF5 specified in the
CMakeLists.txt
if it works with the newest...
Should I create a PR with the version requirement removed?
Are you sure that HDF5 change is needed? That actually should be the minimum required version, according to this CMake output:
-- Found HDF5: /usr/local/Cellar/hdf5/1.14.3/lib/libhdf5.dylib (found suitable version "1.14.3", minimum required is "1.8") found components: C HL
Builds fine on my system with HDF5 1.14.3 plus the change from my PR.
EXACT
is not specified, so it should look for a compatible version, which I assume means 1.x, >= 1.8
Fair point! Not sure why I got build errors before, and now I'm regretting having not kept the log files.
Maybe it wasn't symlinked, or just couldn't find it, similar to Qt6?
Looks like almost everything installed correctly except the python interface. One remaining issue now is that I can't get any of the Tutorials to run.
For example:
MacBook-Pro:Tutorials arko$ python3 Helical_Antenna.py
Traceback (most recent call last):
File "/Users/arko/Downloads/openEMS-Project/openEMS/python/Tutorials/Helical_Antenna.py", line 17, in <module>
from CSXCAD import CSXCAD
File "/Users/arko/Library/Python/3.9/lib/python/site-packages/CSXCAD-0.6.2-py3.9-macosx-10.9-x86_64.egg/CSXCAD/__init__.py", line 19, in <module>
from CSXCAD.CSXCAD import ContinuousStructure
ImportError: dlopen(/Users/arko/Library/Python/3.9/lib/python/site-packages/CSXCAD-0.6.2-py3.9-macosx-10.9-x86_64.egg/CSXCAD/CSXCAD.cpython-39-darwin.so, 0x0002): Library not loaded: @rpath/libCSXCAD.0.dylib
Referenced from: <F05ADA80-B8D1-3887-BF1E-34AFCD495AAA> /Users/arko/Library/Python/3.9/lib/python/site-packages/CSXCAD-0.6.2-py3.9-macosx-10.9-x86_64.egg/CSXCAD/CSXCAD.cpython-39-darwin.so
Reason: no LC_RPATH's found
I originally tried the ./update_openEMS.sh ~/opt/openEMS --python
command, but that seems to have not taken (even though it threw no errors. After looking through the wiki, I figured I'd try manually installing the CSXCAD and openEMS python interfaces using pip3 install . --user
in their respective folders, but I get the following error:
MacBook-Pro:python arko$ pip3 install . --user
Processing /Users/arko/Downloads/openEMS-Project/CSXCAD/python
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [17 lines of output]
Traceback (most recent call last):
File "/Users/arko/Library/Python/3.9/lib/python/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/arko/Library/Python/3.9/lib/python/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/arko/Library/Python/3.9/lib/python/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/private/var/folders/56/nrjnt_7502345jgyjl4c6lvh0000gn/T/pip-build-env-6ly3c5n9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/private/var/folders/56/nrjnt_7502345jgyjl4c6lvh0000gn/T/pip-build-env-6ly3c5n9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/private/var/folders/56/nrjnt_7502345jgyjl4c6lvh0000gn/T/pip-build-env-6ly3c5n9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/private/var/folders/56/nrjnt_7502345jgyjl4c6lvh0000gn/T/pip-build-env-6ly3c5n9/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 3, in <module>
ModuleNotFoundError: No module named 'Cython'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Any ideas what the issue might be? I feel I'm close to getting this all to work!
Your first attempt there appears to have tried to install it into the macOS-supplied python, which didn't end well (some parts didn't install or there's some path issue). I would try to remove what it installed so it doesn't cause any further problems.
You second attempt looks pretty close, and appears you just need Cython, which you should be able to get through Homebrew.
I'd recommend trying to install in a virtual environment. Homebrew's Python gets overwritten when it's upgraded, as does the system Python IIRC. You also need root to install to system Python.
I also have a draft PR here that installs the Python bindings with Homebrew. I believe it worked last time I tried, but it's still a draft since I don't know the best way to depend on Python packages in Homebrew.
This is turning into a doozy. I tried what you suggested with creating a virtual environment. After sourcing it and installing the requirements, I again tried ./update_openEMS.sh ~/opt/openEMS --python
. Although it claimed to be successful, the tutorials now crash with:
(openems_venv) MacBook-Pro:Tutorials arko$ python Simple_Patch_Antenna.py
Traceback (most recent call last):
File "/Users/arko/Downloads/openEMS-Project/openEMS/python/Tutorials/Simple_Patch_Antenna.py", line 17, in <module>
from CSXCAD import ContinuousStructure
ModuleNotFoundError: No module named 'CSXCAD'
So I tried to CSXCAD manually using pip install .
in the CSXCAD/python
and openEMS/python
directories and got it to install (it failed initially, but after installing wheel it worked).
I thought I was home free, but after running (openems_venv) MacBook-Pro:Tutorials arko$ python Simple_Patch_Antenna.py
again, I got:
Traceback (most recent call last):
File "/Users/arko/Downloads/openEMS-Project/openEMS/python/Tutorials/Simple_Patch_Antenna.py", line 17, in <module>
from CSXCAD import ContinuousStructure
File "/Users/arko/Downloads/openEMS-Project/openems_venv/lib/python3.9/site-packages/CSXCAD/__init__.py", line 19, in <module>
from CSXCAD.CSXCAD import ContinuousStructure
File "CSXCAD/CSXCAD.pyx", line 1, in init CSXCAD.CSXCAD
File "CSXCAD/CSProperties.pyx", line 1, in init CSXCAD.CSProperties
File "CSXCAD/CSTransform.pyx", line 26, in init CSXCAD.CSTransform
ImportError: dlopen(/Users/arko/Downloads/openEMS-Project/openems_venv/lib/python3.9/site-packages/CSXCAD/Utilities.cpython-39-darwin.so, 0x0002): Library not loaded: @rpath/libCSXCAD.0.dylib
Referenced from: <8FA88314-D1DF-3834-86C0-44DFF0E7F399> /Users/arko/Downloads/openEMS-Project/openems_venv/lib/python3.9/site-packages/CSXCAD/Utilities.cpython-39-darwin.so
Reason: no LC_RPATH's found
I'm starting to run out of ideas as why I'm struggling to build and run this. Could this be an arm arch issue?
I just set up another branch in my forks of these repos with the two patches I've mentioned combined, and now my brew formula that installed the python bindings no longer works... It's getting rather exhausting trying to keep this working since someone somewhere changes something that breaks the whole build process for an otherwise straightforward C++ project every couple months.
Anyways... maybe try this? Assuming you've installed openEMS through Brew at this point, perhaps it can't find the libraries.
DYLD_LIBRARY_PATH=$(brew --prefix openems)/lib python3 Simple_Patch_Antenna.py
Now we're cooking with fire!
Although the command above did not work exactly as expected (result showed it could not find the lib), it lead me to go hunting for it. After some digging I realized I had installed it in ~/opt/openEMS
. So running the following command works:
DYLD_LIBRARY_PATH=~/opt/openEMS/lib python3 Simple_Patch_Antenna.py
Seems like a good step forward. At least the "debug whac-a-mole" game has finally come to an end, and I can start mess with the tool.
Honestly, I commend you and the team for your efforts. I can only imagine how difficult it must be keeping up with this ever changing dependency codebase and build environment. OpenEMS is a powerful tool and folks like you, who take the time out of their day, to help folks install and use it are the champion of champions. Thank you!
Cool, glad I could be of some help! I'm only really flagged as a contributor because I've been doing this same kind of thing, fixing random weird build issues, primarily on macOS because Brew always uses the latest libraries. I've hardly even interacted with the actual main authors of this project.
Honestly Homebrew doesn't seem to play well with Python packages for some reason, so it's probably best to do what you did + install the main tool from Homebrew once that's fixed.
For reference, as of Sat Mar 9 22:47:38.417652341526546346 PST 2024
, the following worked for me:
% brew untap thliebig/openems
% brew tap aWZHY0yQH81uOYvH/openems "https://github.com/aWZHY0yQH81uOYvH/openEMS-Project.git"
% git -C $(brew --repo awzhy0yqh81uoyvh/openems) checkout testing
% brew install --HEAD openems
% git clone --recurse-submodules -j8 "https://github.com/thliebig/openEMS-Project.git"
% cd openEMS-Project
% python3 -m venv openems_venv
% source openems_venv/bin/activate
% pip install cython matplotlib
% pip install CSXCAD/python
% pip install openEMS/python
% cd openEMS/python/Tutorials
% python3 Simple_Patch_Antenna.py
May need some other Python dependencies, but should be obvious based on the errors.
When the changes get merged, uninstalling my fork and changing back to the official thliebig/openEMS
tap is a good idea.
FWIW - I tried your command list and still needed DYLD_LIBRARY_PATH=~/opt/openEMS/lib python3 Simple_Patch_Antenna.py
to make it work. Wonder what I borked to need it...
I bet it's because you're on ARM and Homebrew doesn't install in /usr/local/lib
, rather /opt/homebrew/lib
. I think the former is a standard place for the system to look for libraries, but the latter is not. Homebrew updates your .zprofile
file to set your $PATH
variable so you can find binaries automatically, but probably doesn't touch $DYLD_LIBRARY_PATH
.
Also could be a matter of whether you're invoking the python3
that Brew installs (might already know about /opt/homebrew/lib
?) vs. the system's Python. I'm on an Intel Mac so I don't really know if that's a thing or not.
Also if you still need DYLD_LIBRARY_PATH=~/opt/openEMS/lib
, that means you're probably using the openEMS you installed manually. Does it work if you try DYLD_LIBRARY_PATH=/opt/homebrew/lib
?
If you would like, I think I fixed the Brew formula that should automatically install the Python bindings. Works on my Intel Mac and an M1 machine.
It's on the testing2
branch of my openEMS-Project
fork.
Uninstall old openEMS and remove official tap if you haven't already
% brew uninstall openems
% brew untap thliebig/openems
Tap my fork and check out the testing2
branch
% brew tap aWZHY0yQH81uOYvH/openems "https://github.com/aWZHY0yQH81uOYvH/openEMS-Project.git"
% git -C $(brew --repo awzhy0yqh81uoyvh/openems) checkout testing2
% brew install --HEAD openems
Try Python bindings, this time without any manual installation or virtual environments
% git clone --recurse-submodules -j8 "https://github.com/thliebig/openEMS-Project.git"
% cd openEMS-Project/openEMS/python/Tutorials
% python3 Simple_Patch_Antenna.py
(assuming the first python3
in your PATH
is the one provided by Homebrew)
If this works, I'll update my PR (https://github.com/thliebig/openEMS-Project/pull/169) and hopefully that can get merged.
Having been using openEMS for several months, I tried re-installing it on my Mac, using OSX14.4, with similar results to you. What I found was that the hdf5 currently used by homebrew (1.14.3) seems to break almost everything - https://github.com/Homebrew/homebrew-core/issues/159691
My workaround for this was to do - brew unlink hdf5 brew install hdf5@1.10 brew link hdf5@1.10 --force
which gets things compiling, but not installing. I have attached the log file, as I'm not sure where to go from here.
I will try the manual install process outlined above, hopefully it will work. Before I broke things openEMS was working really well - I was able to compare results of a planar transformer with the same design in Keysight Momentum, and the inductance differences were in the order 0.2%, and k was better than that. 02.make.txt
@valgamaa I'm not sure why messing with the hdf5 version affects anything. Those errors you were running into with the Homebrew install are related to the Boost Thread library, which should be fixed by this PR: https://github.com/thliebig/openEMS/pull/137
My testing branch I mentioned above includes that fix, as well as a new method for installing the Python bindings.
It compiles and installs fine for me with the default hdf5 (1.14.3) that is referenced by the current Brew formula, but perhaps you mean it doesn't work when trying to use certain features of openEMS?
I have found that installing on Mac using methods outside the recommended Homebrew formula can be particularly temperamental.
@valgamaa I'm not sure why messing with the hdf5 version affects anything. Those errors you were running into with the Homebrew install are related to the Boost Thread library, which should be fixed by this PR: thliebig/openEMS#137
This is the log file after changing the hdf5 version. Before, I just got the error referenced in the link I posted, and didn't get any further.
My testing branch I mentioned above includes that fix, as well as a new method for installing the Python bindings.
I'm trying that now, looking good so far (but it hasn't completed yet).
It compiles and installs fine for me with the default hdf5 (1.14.3) that is referenced by the current Brew formula, but perhaps you mean it doesn't work when trying to use certain features of openEMS?
I have found that installing on Mac using methods outside the recommended Homebrew formula can be particularly temperamental.
Yes, I try to stick with 'vanilla' Homebrew as I'm not very knowledgeable about all of this.
@aWZHY0yQH81uOYvH I can confirm that your branch works with OSX 14.4. I'm going to revert the links to hdf5 now before I forget..
Thanks for your efforts, I wouldn't have been able to solve the boost problem myself.
Nice! Also don't forget to un-tap my branch and switch back to the official one once those PRs get merged eventually.
I'm running into build issues while installing OpenEMS using the Installing Ready-Made Packages for macOS.
Has anyone seen the following build error?
Machine: MacBook Pro M3 Max macOS: Sonoma 14.1.2
Build error occurs while running
brew install --HEAD openems
: