search

prashnts / pybloomfiltermmap3

Fast Python Bloom Filter using Mmap
https://github.com/prashnts/pybloomfiltermmap3
MIT License
126 stars 22 forks source link

Make in-memory bloomfilters pickle-able #43

Open stevesimmons opened 3 years ago

stevesimmons commented 3 years ago

The bloomfilters don't have reduce and setstate functions and so cannot be pickled. In particular, this prevents in-memory bloomfilters being used in multiprocessing.Pool or in cloud lambda functions, etc.

Do you have any plans to add these?

I guess the easiest way to do it is adding cython functions to read/write _bf.array.

stevesimmons commented 3 years ago

I had a go at solving this here: https://github.com/prashnts/pybloomfiltermmap3/pull/44