Python bindings doesn't work with Python 3.8+

[maresmar]

When I tried to install ufal.nametag I got the following error with Python 3.8 (with Python 3.7 it works fine).

    Running setup.py install for ufal.nametag: started
    Running setup.py install for ufal.nametag: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /srv/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-afntpbwc/ufal-nametag/setup.py'"'"'; __file__='"'"'/tmp/pip-install-afntpbwc/ufal-nametag/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-bzanm4ur/install-record.txt --single-version-externally-managed --compile --install-headers /srv/venv/include/site/python3.8/ufal.nametag
         cwd: /tmp/pip-install-afntpbwc/ufal-nametag/
    Complete output (23 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/ufal
    copying ufal/__init__.py -> build/lib.linux-x86_64-3.8/ufal
    copying ufal/nametag.py -> build/lib.linux-x86_64-3.8/ufal
    running build_ext
    building 'ufal_nametag' extension
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/nametag
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Inametag/include -I/srv/venv/include -I/usr/local/include/python3.8 -c nametag/nametag.cpp -o build/temp.linux-x86_64-3.8/nametag/nametag.o -std=c++11 -fvisibility=hidden -w
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Inametag/include -I/srv/venv/include -I/usr/local/include/python3.8 -c nametag/nametag_python.cpp -o build/temp.linux-x86_64-3.8/nametag/nametag_python.o -std=c++11 -fvisibility=hidden -w
    nametag/nametag_python.cpp: In function ‘void SwigPyStaticVar_dealloc(PyDescrObject*)’:
    nametag/nametag_python.cpp:3321:3: error: ‘_PyObject_GC_UNTRACK’ was not declared in this scope
       _PyObject_GC_UNTRACK(descr);
       ^~~~~~~~~~~~~~~~~~~~
    nametag/nametag_python.cpp:3321:3: note: suggested alternative: ‘PyObject_GC_UnTrack’
       _PyObject_GC_UNTRACK(descr);
       ^~~~~~~~~~~~~~~~~~~~
       PyObject_GC_UnTrack
    error: command 'gcc' failed with exit status 1
    ----------------------------------------

I found a similar issue in UDPipe that is probably solved by commit 86f1c8c171. Maybe it could help ;-)

[foxik]

Yup, you are right. I know about it for some time, but teaching in the current situation is taking all my time and I have little left for this. But I will try to release an updated version whenever I can. (I already have an infrastructure for building binary wheels, already applied on ufal.morphodita, so binary wheels will be part of the next release.)

BTW, you can make the Python bindings from the sources (in the bindings/python directory) in the meantime.

Leaving open for tracking.

Cheers!

[starenka]

Hi @foxik, any news about the fix/whl release? Cheers.

[crabhi]

@maresmar I was able to patch the existing release with this little change.

--- a/nametag/nametag_python.cpp    2016-09-22 14:33:51.000000000 +0200
+++ b/nametag/nametag_python.cpp    2021-11-30 14:45:57.096869000 +0100
@@ -3318,7 +3318,7 @@

 SWIGINTERN void
 SwigPyStaticVar_dealloc(PyDescrObject *descr) {
-  _PyObject_GC_UNTRACK(descr);
+  PyObject_GC_UnTrack(descr);
   Py_XDECREF(PyDescr_TYPE(descr));
   Py_XDECREF(PyDescr_NAME(descr));
   PyObject_GC_Del(descr);

The process is basically:

pip download ufal.nametag==1.1.2.1
tar -xf *
cd ufal.nametag*
patch -p1 < $patch_above
pip install .

[crabhi]

Or, if you download the repo, then with SWIG installed, the bindings build cleanly. I faced a strange problem when trying to use string vectors between nametag and morphodita.

Basically, sending morphodita.Forms into nametag works with SWIG 3 but not with SWIG 4. To avoid adding conversions in Python, I went with the hack above to stay at SWIG 3.

[foxik]

BTW, I just did a maintenance release of NameTag 1, including Python wheels of ufal.nametag 1.2.0.1, for Python 3.6-3.11, Linux 32bit&64bit&aarch64; Windows 32bit&64bit; macOS Intel&ARM.