17.0.0 -> 18.0.0: coincurve/_libsecp256k1.cpython-39.so: Undefined symbol "secp256k1_schnorrsig_sign32"

[hhartzer]

Hi!

I'm not sure if coincurve can use libsecp256k1 libraries on the system or not, but after upgrading from coincurve 17.0.0 to 18.0.0, I'm seeing this error when I import coincurve:

coincurve/_libsecp256k1.cpython-39.so: Undefined symbol "secp256k1_schnorrsig_sign32"

I am not sure if this is worth working around or not, but others may run into the same.

I am on FreeBSD 12 and have had this so far with Python 3.9 and 3.10.

Thanks!

[ofek]

How did you install?

[hhartzer]

python3.9 -m venv venv3.9
venv3.9/bin/python -m pip install coincurve
venv3.9/bin/python -c 'import coincurve'

Also did that with 3.10. Initially, I noticed it while using Pipenv.

[ofek]

❯ docker run --rm -it python:3.9 bash
root@302946579f2d:/# python -m pip install coincurve
Collecting coincurve
  Downloading coincurve-18.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.4/1.4 MB 7.4 MB/s eta 0:00:00
Collecting cffi>=1.3.0
  Downloading cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.2/441.2 KB 9.4 MB/s eta 0:00:00
Collecting asn1crypto
  Downloading asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.0/105.0 KB 6.0 MB/s eta 0:00:00
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 7.0 MB/s eta 0:00:00
Installing collected packages: asn1crypto, pycparser, cffi, coincurve
Successfully installed asn1crypto-1.5.1 cffi-1.15.1 coincurve-18.0.0 pycparser-2.21
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
root@302946579f2d:/# python -c 'import coincurve'

[hhartzer]

I am wondering if this is only on FreeBSD or only when the wheels are built locally. I'm trying to test that now with ./venv3.9/bin/python -m pip install coincurve==18.0.0 --no-binary ":all:" on a Debian system.

[ofek]

Try COINCURVE_IGNORE_SYSTEM_LIB https://ofek.dev/coincurve/install/#source

[hhartzer]

Good suggestion. It did not work for me, though.

zsh# COINCURVE_IGNORE_SYSTEM_LIB=1 ./venv3.9/bin/pip install coincurve --no-cache-dir
Collecting coincurve
  Downloading coincurve-18.0.0.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 1.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cffi>=1.3.0 in ./venv3.9/lib/python3.9/site-packages (from coincurve) (1.15.1)
Requirement already satisfied: asn1crypto in ./venv3.9/lib/python3.9/site-packages (from coincurve) (1.5.1)
Requirement already satisfied: pycparser in ./venv3.9/lib/python3.9/site-packages (from cffi>=1.3.0->coincurve) (2.21)
Building wheels for collected packages: coincurve
  Building wheel for coincurve (pyproject.toml) ... done
  Created wheel for coincurve: filename=coincurve-18.0.0-cp39-cp39-freebsd_12_3_release_p8_amd64.whl size=32974 sha256=f3856a403e2275f7877b7c9cbd70b05bed006352e3db1f545e9927123da503c9
  Stored in directory: /tmp/pip-ephem-wheel-cache-lz84k8ku/wheels/83/d3/47/f4560ce8ae386340055087ff2201b0f7ae1ef988fe1c6b09ea
Successfully built coincurve
Installing collected packages: coincurve
Successfully installed coincurve-18.0.0
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/root/tmp/venv3.9/bin/python3.9 -m pip install --upgrade pip' command.
zsh# COINCURVE_IGNORE_SYSTEM_LIB=1 ./venv3.9/bin/python -c 'import coincurve'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/root/tmp/venv3.9/lib/python3.9/site-packages/coincurve/__init__.py", line 1, in <module>
    from coincurve.context import GLOBAL_CONTEXT, Context
  File "/root/tmp/venv3.9/lib/python3.9/site-packages/coincurve/context.py", line 4, in <module>
    from coincurve.flags import CONTEXT_ALL, CONTEXT_FLAGS
  File "/root/tmp/venv3.9/lib/python3.9/site-packages/coincurve/flags.py", line 1, in <module>
    from ._libsecp256k1 import lib
ImportError: /root/tmp/venv3.9/lib/python3.9/site-packages/coincurve/_libsecp256k1.cpython-39.so: Undefined symbol "secp256k1_schnorrsig_sign32"

[hhartzer]

Doing the build from source on Debian 11 was successful, so just FreeBSD so far. I do have a secp256k1 package installed on FreeBSD, though.

[ofek]

add --ignore-installed?

[hhartzer]

Same error with --ignore-installed.

[ofek]

cc @darosior

[darosior]

I can't manage to reproduce it on a Debian system under various conditions. I've skimmed the libsecp sources (that i'm not familiar with) looking for BSD-specific exceptions and couldn't find any.

[hhartzer]

I have no idea why, but this started working for me. Possible there was a package update that fixed it, so likely not a coincurve bug. Sorry about that.

Thank you for your help!