search

darvid / python-hyperscan

🐍 A CPython extension for the Hyperscan regular expression matching library.
https://python-hyperscan.readthedocs.io/en/latest/
MIT License
168 stars 29 forks source link

multiprocessing problem. #45

Open jerzyorlowskimim opened 2 years ago

jerzyorlowskimim commented 2 years ago

I try to run pileline using mutliprocessing with hyperscan.

i use with closing(Pool(args.nr_proc)) as pool: to define process_pool I use pool.apply_async() to generate list of testks

When I run tasks I get:


    raise self._value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 537, in _handle_tasks
    put(task)
  File "/usr/lib/python3.9/multiprocessing/connection.py", line 211, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
TypeError: cannot pickle 'hyperscan.Database' object```