Convex-Labs / honestnft-shenanigans

HonestNFT Shenanigan Scanning Tools
https://honestnft-shenanigans.readthedocs.io/
MIT License
176 stars 70 forks source link

fix: log ChunkedEncodingError and retry #103

Closed Barabazs closed 2 years ago

Barabazs commented 2 years ago

The script often throws an error when the connection is closed or the server returns an unexpected response.

Traceback (most recent call last):
  File "honestnft-shenanigans/fair_drop/suspicious.py", line 234, in <module>
    main(
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "honestnft-shenanigans/fair_drop/suspicious.py", line 150, in main
    results = p.starmap(is_nft_suspicious, batch)
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 372, in starmap
    return self._map_async(func, iterable, starmapstar, chunksize).get()
  File "/usr/lib/python3.10/multiprocessing/pool.py", line 771, in get
    raise self._value
multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7faa96b3ac20>'. Reason: 'TypeError("cannot pickle 'zlib.Decompress' object")'

With this PR the script will simply log the error (and the URL) and retry, without exiting.