Closed earandel closed 3 years ago
Synss
commented
3 years ago Hi @earandel and thank you for your report.
I actually develop on MacOS (Catalina) so I know it works on that platform. However, I do not package for MacOS. For that, you should probably see with the MacPorts guys.
Let me have a look anyway. I can probably tell you what is wrong ;)
Cheers, Mathias
Synss
commented
3 years ago Note: MacOS is not supported--so this is not a bug.
Synss
commented
3 years ago I install everything from source using the scripts under scripts. They are documented.
Maybe simpler in your case (It seemed to work--but keep in mind that my system is tainted because I develop here):
# Get a local copy of the sources
git clone https://github.com/Synss/python-mbedtls.git pymbedtls
cd pymbedtls
# Create a virtual environment where you can install python-mbedtls
python3 -m venv venv
. ./venv/bin/activate
pip install cython
# You probably already have that next one
port install mbedtls
# Now, you need the path where MacPorts installs things on your system: the MacPorts root
port contents mbedtls
# $libdir is the MacPorts root: the part of the path up to .../include/mbedtls or .../lib/mbedtls, etc. above
libdir=PATH_TO_THE_MACPORTS_ROOT
C_INCLUDE_PATH="$libdir/include" LIBRARY_PATH="$libdir/lib" DYLD_LIBRARY_PATH="/usr/lib:$libdir/lib" python setup.py installBut you should really contact MacPorts maintainers or contribute a port. I'd gladly help.
earandel
commented
3 years ago Thank you very much for this... the build nearly worked with the above. It failed with the following error:
build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_platform.c:608:10: fatal error:
'mbedtls/platform_util.h' file not found
#include "mbedtls/platform_util.h"
^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.I worked around it by creating a symlink in the venv/include dir to the mbedtls directory in the ports include directory. I'm not sure why it failed on this particular header file because it got quite a ways through the build steps before hitting this error so I'm guessing it was able to include some of the mbedtls header files. Anyway, after that change and the above instructions from you, I was able to complete the build and installation in the virtualenv. The import of the mbedtls works now so hopefully everything will work now... I have some other dependency issues to work out with some other python modules before I can test further.
I did try the virtualenv approach before as well but didn't install cython or setup the above env vars so ended up with the same result as installing on the system.
If macOS is not officially support, this issue can be closed then... I mistook the comment under the Installation section mentioning testing on Linux and macOS to mean the platforms were supported. Even if not officially supported, your unofficial support was fantastic in resolving my issue! :) Thanks again.
Synss
commented
3 years ago If macOS is not officially support, this issue can be closed then...
macOS is tested and I would fix bugs quickly. I just do not do packaging—because I believe most people interested in this library use MacPorts or Homebrew. I guess it would not be very difficult to build wheel for that platform. Maybe after I refactored my TravisCI config.
Synss
commented
3 years ago Hi! Issue #36 reported missing files in the archive. That could very well be the cause for the failing installation.
Synss
commented
3 years ago Hi! It was indeed a bug and should now be fixed in 1.4.1. Could you please try once more and report here? Thank you!
earandel
commented
3 years ago Ok, I tried installing without the virtualenv. It got quite a ways into the build but failed trying to find mbedtls/platform_util.h which is in the /opt/local/include directory installed by MacPorts. Interestingly enough, it did find the dynamic library early on in the build process now which is in the /opt/local/lib directory so this is an improvement. I had to manually work around the include file issue before when installing manually using the virtualenv so something must not be quite right with the include rules since it isn't adding /opt/local/include to the compile command.
# pip3 install python-mbedtls
Collecting python-mbedtls
Downloading python-mbedtls-1.4.1.tar.gz (121 kB)
|################################| 121 kB 786 kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Collecting certifi
Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
|################################| 155 kB 828 kB/s
Building wheels for collected packages: python-mbedtls
Building wheel for python-mbedtls (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /opt/local/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /opt/local/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp_ox3i77p
cwd: /private/tmp/pip-install-6kztug_3/python-mbedtls
Complete output (53 lines):
loading: '/opt/local/lib/libmbedtls.dylib'
mbedtls version: mbed TLS 2.24.0
running bdist_wheel
running build
running build_py
creating build
creating build/3.6.12
creating build/3.6.12/lib.macosx-10.15-x86_64-3.6
creating build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls
copying src/mbedtls/hashlib.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls
copying src/mbedtls/__init__.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls
copying src/mbedtls/secrets.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls
copying src/mbedtls/hmac.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls
creating build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/ARIA.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/ARC4.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/AES.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/__init__.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/DES3.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/Camellia.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/CHACHA20.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/Blowfish.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/DES3dbl.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
copying src/mbedtls/cipher/DES.py -> build/3.6.12/lib.macosx-10.15-x86_64-3.6/mbedtls/cipher
running build_ext
cythoning src/mbedtls/_platform.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_platform.c
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls
cythoning src/mbedtls/mpi.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/mpi.c
cythoning src/mbedtls/pk.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/pk.c
cythoning src/mbedtls/tls.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/tls.c
cythoning src/mbedtls/version.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/version.c
cythoning src/mbedtls/_random.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_random.c
cythoning src/mbedtls/hkdf.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/hkdf.c
cythoning src/mbedtls/x509.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/x509.c
cythoning src/mbedtls/exceptions.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/exceptions.c
cythoning src/mbedtls/_ringbuf.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_ringbuf.c
cythoning src/mbedtls/_md.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_md.c
cythoning src/mbedtls/cipher/_cipher.pyx to build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/cipher/_cipher.c
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/cipher
building 'mbedtls._platform' extension
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/build
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/build/3.6.12
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/build/3.6.12/temp.macosx-10.15-x86_64-3.6
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex
creating build/3.6.12/temp.macosx-10.15-x86_64-3.6/build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_platform.c -o build/3.6.12/temp.macosx-10.15-x86_64-3.6/build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_platform.o
build/3.6.12/temp.macosx-10.15-x86_64-3.6/pyrex/mbedtls/_platform.c:608:10: fatal error: 'mbedtls/platform_util.h' file not found
#include "mbedtls/platform_util.h"
^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for python-mbedtls
Failed to build python-mbedtls
ERROR: Could not build wheels for python-mbedtls which use PEP 517 and cannot be installed directly
# ls -l /opt/local/include/mbedtls/platform_util.h
-rw-r--r-- 1 root wheel 7727 Aug 27 04:31 /opt/local/include/mbedtls/platform_util.hNow, the easiest way for you would be to find a simple MacPorts Python port and adapt it to python-mbedtls. You probably only have to change the filename and version and maybe a couple of links and checksums.
Anyway, compiling from source, you will need to tell clang where the headers and the libraries are. You can check at my (shell script)[https://github.com/Synss/python-mbedtls/blob/master/scripts/install-pymbedtls.sh] for how I do that during development.
I also use MacPorts but I install it locally so the following paths may not be entirely correct.
Anyway, for the compilation, you will need at least C_INCLUDE_PATH and LIBRARY_PATH and to use the library, you will need DYLD_LIBRARY_PATH.
So in your case, probably
export C_INCLUDE_PATH=/opt/local/include
export LIBRARY_PATH=/opt/local/lib
export DYLD_LIBRARY_PATH=$LIBRARY_PATH
pip3 install python-mbedtls
# ...
# python3
# import mbedtlsBut really, you should do it with MacPorts ;)
Synss
commented
3 years ago I could write a port but I do not want to submit it or maintain it (because of my limited time–nothing else). Would you be interested in maintaining it for MacPorts?
I would write an initial, working port and the maintenance burden would probably be limited to checking for your releases, updating the version, and reaching out to me when it does not work.
earandel
commented
3 years ago Strangely enough, if I set all the env vars you suggested above, the installation fails with an error that it cannot find the _iconv symbol in libiconv.2.dylib. However, if I just do
export C_INCLUDE_PATH=/opt/local/include
pip3 install python-mbedtls
# ...
# python3
# import mbedtlsEverything completes successfully.
As for maintaining a port for this, let me read up on what is involved in maintaining a port. I don't have a lot of extra time but it might not be too time consuming or difficult.
Synss
commented
3 years ago Great that it finally worked for you as well!
crifan
commented
8 months ago Have fixed: mbedtls/_platform.c:1205:10: fatal error: 'mbedtls/platform_util.h' file not found
git clone https://github.com/Mbed-TLS/mbedtls.gitmake and make install/usr/localFinal worked:
export C_INCLUDE_PATH=/usr/local/include
export LIBRARY_PATH=/usr/local/lib/
pip3 install python-mbedtls
second-string
commented
6 months ago Don't forget to pip install the requirements for mbedtls if it's not installing properly. Before running make and make install:
cd mbedtls
python3 -m venv .venv
source .venv/bin/activate
pip3 install -r scripts/basic.requirements.txt
I am submitting a …
Description
The latest version of python-mbedtls 1.4.0 does not seem to install correctly on macOS Catalina 10.15.7. After attempting to install multiple times via pip3 or from source using Apple's version of pip3/python3 or a macports selected version of pip3/python3, the package always is missing the dynamically loaded shared objects in the medbtls python directory. This prevents importing of the mbedtls package. I have a working version of the same package on Ubuntu 18.04 LTS which includes the missing objects but am not sure how to get the package working on macOS.
Current behavior
After installing mbedtls (from MacPorts) either using pip3 to install python-mbedtls or installing from the source package (as in the egg example below), the mbedtls package cannot be imported in a python script as follows:
Here are the contents of the egg referenced above:
The same occurred when trying to install using apple's python3. Here are the contents of the mbedtls directory after installation:
Here are the contents of the build directory when building from source:
Expected behavior
It looks like the there are a number of modules that should be loaded dynamically as shared objects but for some reason these are not being built/installed. On Ubuntu 18.04LTS, here is how things look with a working installation:
Steps to reproduce
Minimal demo of the problem
Other information
macOS Catalina v10.15.7 pip 20.2.4 Python 3.6.12 mbedtls @2.24.0