DinoTools / python-ssdeep

Python wrapper for ssdeep fuzzy hashing library
GNU Lesser General Public License v3.0
152 stars 30 forks source link

Multithread error #50

Closed meir555 closed 4 years ago

meir555 commented 4 years ago
ISSUE TYPE
Python ssdeep VERSION
ssdeep 3.4
OS / ENVIRONMENT

OS:

Python version:

ssdeep lib:

SUMMARY

When using ssdeep with threading, I get an error UserWarning: reimporting '_ssdeep_cffi_a28e5628x27adcb8d' might overwrite older definitions

STEPS TO REPRODUCE

hash1 = '123:123123123:123123123' hash2 = '123:456456456:456456456'

def my_thread_1(): for i in range(1000): ssdeep.compare(hash1, hash2)

t1 = threading.Thread(target=my_thread_1) t2 = threading.Thread(target=my_thread_1)

t1.start() t2.start()

t1.join() t2.join()

print('done example 1')


- **Example 2**

import ssdeep, threading

text = 'blabla'

def my_thread_2(): for i in range(1000): ssdeep.hash(text)

t1 = threading.Thread(target=my_thread_2) t2 = threading.Thread(target=my_thread_2)

t1.start() t2.start()

t1.join() t2.join()

print('done example 2')


##### EXPECTED RESULTS
Code should run without errors

##### ACTUAL RESULTS

Warning (from warnings module): File "/home/user/.local/lib/python3.8/site-packages/cffi/vengine_cpy.py", line 191 warnings.warn("reimporting %r might overwrite older definitions" UserWarning: reimporting '_ssdeep_cffi_a28e5628x27adcb8d' might overwrite older definitions

Same issue with python 2.7.18:

/usr/local/lib/python2.7/dist-packages/cffi/vengine_cpy.py:192: UserWarning: reimporting '_ssdeep_cffi_a28e5628x27adcb8d' might overwrite older definitions % (self.verifier.get_module_name()))

phibos commented 4 years ago

Thanks for reporting the issue. I could reproduce the error with Python 3.8.5.

phibos commented 4 years ago

Back in 2014 when I have migrated the library to use cffi most Linux distributions shipped with python-cffi < 1.0. Since cffi 1.0 the verify() function has been marked as deprecated.

if your CFFI project uses ffi.verify() [...] then you should really rewrite it [...]

I think we should follow the advice and rewrite the code before we try to fix the issue. This may take a few days.

phibos commented 4 years ago

Feel free to have a look at PR #51

meir555 commented 3 years ago

Thanks @phibos The package number in pip is still 3.4, can you update the number to 3.5 so that I can get the new version using pip install?