Symbol not found: __PyThreadState_Current

[aneksteind]

In an attempt to run the Sycamore depth 12 example, I've run into the following exception:

Traceback (most recent call last):
  File "test_cotengra.py", line 40, in <module>
    info = tn.contract(all, optimize=opt, get='path-info', output_inds=[])
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/quimb/tensor/tensor_core.py", line 3141, in contract
    return tensor_contract(*self, **opts)
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/quimb/tensor/tensor_core.py", line 315, in tensor_contract
    path_info = get_contraction(eq, *ops, path=True, **contract_opts)
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/quimb/tensor/tensor_core.py", line 110, in get_contraction
    return fn(eq, *shapes, **kwargs)
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/quimb/tensor/tensor_core.py", line 78, in _get_contract_path
    return oe.contract_path(eq, *shapes, shapes=True, **kwargs)[1]
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/opt_einsum/contract.py", line 259, in contract_path
    path = path_type(input_sets, output_set, dimension_dict, memory_arg)
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/cotengra/hyper.py", line 347, in __call__
    for trial in trials:
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/tqdm/std.py", line 1129, in __iter__
    for obj in iterable:
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/cotengra/hyper.py", line 312, in _gen_results_parallel
    yield self.get_and_report_next_future()
  File "/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/cotengra/hyper.py", line 296, in get_and_report_next_future
    trial = future.result()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
ImportError: dlopen(/Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/kahypar.cpython-37m-darwin.so, 2): Symbol not found: __PyThreadState_Current
  Referenced from: /Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/kahypar.cpython-37m-darwin.so
  Expected in: flat namespace
 in /Users/david/Develop/cotengra/tensor/lib/python3.7/site-packages/kahypar.cpython-37m-darwin.so

Helpful info: OS: macOS Catalina $PYTHONPATH: /Users/david/Library/Python/3.7/lib/python

I've tried changing my environment variables and editing the CMake of kahypar such that CC=clang, CXX=clang++ and, as a sanity check, to CC=gcc, CXX=g++ -- taking care to uninstall and reinstall the kahypar python bindings with pip each time.

Any insight would be much appreciated.

[jcmgray]

Yes looks like some kahypar build related issue. I've just been using the pypi (pip install) version of kahypar on linux. Can you see if kahypar's own tests run? If not might be a issue to raise over there.

[aneksteind]

I also installed using pypi, I will try and separate the two and update with any progress to rule out building kahypar from source

[aneksteind]

@jcmgray I was able to get the example working on an Ubuntu VM, perhaps this is a macOS thing. I tried running the example from a fresh virtual environment on macOS and got the same error, this time without the kahypar source involved and only cotengra and quimb

[jcmgray]

Hmm, yes sorry I don't have access to any mac to test. Does this minimal python kahypar example in their readme run?

import os
import kahypar as kahypar

num_nodes = 7
num_nets = 4

hyperedge_indices = [0,2,6,9,12]
hyperedges = [0,2,0,1,3,4,3,4,6,2,5,6]

node_weights = [1,2,3,4,5,6,7]
edge_weights = [11,22,33,44]

k=2

hypergraph = kahypar.Hypergraph(num_nodes, num_nets, hyperedge_indices, hyperedges, k, edge_weights, node_weights)

context = kahypar.Context()
context.loadINIconfiguration("<path/to/config>/km1_kKaHyPar_dissertation.ini")

context.setK(k)
context.setEpsilon(0.03)

kahypar.partition(hypergraph, context)

(you would have to download one of the .ini files to run this - cotengra keeps a copy of them)

[aneksteind]

@jcmgray even just the import causes the same difficulty, unfortunately

[aneksteind]

Seems to be related to the kahypar pybind dependency, not a cotengra issue: https://github.com/kahypar/kahypar/issues/58

[jcmgray]

Glad you were able to get it fixed- thanks for keeping this issue updated!