Build for Thread (EC-J-PAKE support)

[edmont]

Hi, I'm trying to build python-mbedtls with EC-J_PAKE support for Thread usage. ARM provides a special configuration file which allows to cover those minimal requirements. I tried replacing the default config and resulted in a lot of errors.

At the moment it's OK for me to have incremental support (not minimal) and I think that can be achieved by defining MBEDTLS_USER_CONFIG_FILE as the config-thread.h.

So I introduced these changes: https://github.com/edmont/python-mbedtls/commit/bd4afe2805579a379b99e3822aeab80f5d6d7e12

But when trying to build (python3 setup.py build) I get this warning: build-3.7.3/temp.linux-x86_64-3.7/pyrex/mbedtls/tls.c:16957:10: warning: implicit declaration of function ‘mbedtls_ssl_set_hs_ecjpake_password’; did you mean ‘mbedtls_ssl_get_record_expansion’? [-Wimplicit-function-declaration]

What am I missing?

[Synss]

Actually, the library is compiled in two steps. First, I compile libmbedtls with scripts/install-mbedtls.sh and then the Python library with the setup.py. As far as I know, EC J-PAKE is not compiled in libmbedtls by default so that it is not available to the Python library.

With scripts/install-mbedtls.sh, you need at least

diff --git a/scripts/install-mbedtls.sh b/scripts/install-mbedtls.sh
index 726dbff..b0ddeab 100755
--- a/scripts/install-mbedtls.sh
+++ b/scripts/install-mbedtls.sh
@@ -45,7 +45,9 @@ fi
 mkdir build
 cd build

-CFLAGS="-DMBEDTLS_ARIA_C=ON" \
+CFLAGS=\
+" -DMBEDTLS_ARIA_C=ON"\
+" -DMBEDTLS_ECJPAKE_C=ON"\
+"" \
 SHARED="ON" \
 make -C .. -j lib
 make -C .. -j install

before you can do anything on the Python side.

What you have at

                ] if coverage else [],
                    extra_compile_args=["-I%s" % configs,
                        "-DMBEDTLS_USER_CONFIG_FILE=<config-thread.h>"]
                )

cannot add features to libmbedtls anymore.

On a Unix-like OS, nm(1) is useful to check whether the symbols are actually compiled in.

Does that answer your question?

[edmont]

I see, I skipped that part. So I just tried the following:

• Add this line to install-mbedtls.sh: " -I$PWD/../configs -DMBEDTLS_USER_CONFIG_FILE='<config-thread.h>'"\
• Run the script so that the library with the desired changes gets compiled and installed. ./scripts/install-mbedtls.sh 2.16.1
• Remove my changes in setup.py.
• Install the Python module: cd python-mbedtls && python3 -m pip install .

But now I get this:

python3
Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mbedtls
Segmentation fault

[edmont]

Oops, I tried the procedure without any of my changes in the Cpython (straight git clone) and it seems something is broken:

• For 2.16.1: same Segmentation fault.
• For 2.7.9: Cpython code won't build: error: ‘MBEDTLS_ECP_DP_CURVE448’ undeclared (first use in this function)

My system is:

$ uname -a
Linux linuxdev 4.19.0-4-amd64 #1 SMP Debian 4.19.28-2 (2019-03-15) x86_64 GNU/Linux

Do you have an identified version of mbedtls for which the module will build without problems?

[edmont]

OK, I made some progress, the segmentation fault is gone after I tried again after removing all contents in:

• /usr/local/src
• /usr/local/include
• /usr/local/lib

But now I get this error:

>>> import mbedtls
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.7/site-packages/mbedtls/__init__.py", line 8, in <module>
    import mbedtls.cipher as cipher
  File "/home/user/.local/lib/python3.7/site-packages/mbedtls/cipher/__init__.py", line 11, in <module>
    from ._cipher import *
ImportError: libmbedcrypto.so.3: cannot open shared object file: No such file or directory

[stepheny]

2.7.9 lacks curve448 support, 2.16.x should be preferred to new projects.

• Install the Python module: cd python-mbedtls && python3 -m pip install .

Is this executed in a fresh cloned git repo or a repo with prior built binary? FYI, cython project builds in stages, updated source code would trigger cython to rebuilt library, but updated system header might not. A manual rebuilt might be needed. Mismatched library version is great source of segfaults.

[stepheny]

After you tried again, could find libmbedcrypto.so libmedcrypto.so.3 in your /usr/local/lib?

[edmont]

@stepheny yes, both of them.

[stepheny]

import ctypes
lib = ctypes.CDLL('libmbedcrypto.so.3')
print(repr(lib))

Will this trigger some error?

[edmont]

Yes:

$ python3
Python 3.7.3rc1 (default, Mar 13 2019, 11:01:15)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> lib = ctypes.CDLL('libmbedcrypto.so.3')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.7/ctypes/__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libmbedcrypto.so.3: cannot open shared object file: No such file or directory
>>> print(repr(lib))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'lib' is not defined

[stepheny]

What about in bash readelf -Ws /usr/local/lib/libmbedcrypto.so? The result is not important. Will this trigger some error?

What about LD_LIBRARY_PATH=/usr/local/lib python and then import mbedtls?

[edmont]

First one, no errors. Second one, neither (python3actually).

[stepheny]

Then, could your code execute as expected in LD_LIBRARY_PATH=/usr/local/lib python3? If so, you may export LD_LIBRARY_PATH=/usr/local/lib in current bash session as a hack.

[edmont]

Yeah, that would make the trick by the moment. Thanks!

[Synss]

Just for the record, the canonical version is the one distributed by travis-ci.

It is documented in the ChangeLog and yes, I should update the README 😅.

@stepheny: Thank you for your help!