TypeError: cannot pickle '_thread.lock' object

[M0dred]

Traceback (most recent call last): File "sublist3r.py", line 1013, in interactive() File "sublist3r.py", line 1010, in interactive res = main(domain, threads, savefile, ports, silent=False, verbose=verbose, enable_bruteforce=enable_bruteforce, engines=engines) File "sublist3r.py", line 952, in main enum.start() File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen return Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle '_thread.lock' object

Python 3.8.2

[rebelnn]

Getting the same as OP -- Python 3.8.0

[amustaque97]

Hey @M0dred & @rebelnn, sorry for the late reply. I'm unable to reproduce the issue, I suggest you update python to the latest version as there is no issue with the thread locking.

If the issue is resolved please mark it as close. Thank You.

[hassankhan98]

my python version is 3.8.3 but still not working?

[christopherowen]

Python 3.8.3 (the latest available in macOS homebrew) experiences this.

% python3 --version
Python 3.8.3

sublist3r % python3 sublist3r.py -d starlink.com

(...)

Traceback (most recent call last):
  File "sublist3r.py", line 1013, in <module>
    interactive()
  File "sublist3r.py", line 1010, in interactive
    res = main(domain, threads, savefile, ports, silent=False, verbose=verbose, enable_bruteforce=enable_bruteforce, engines=engines)
  File "sublist3r.py", line 952, in main
    enum.start()
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/usr/local/Cellar/python@3.8/3.8.3_2/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

[christopherowen]

https://github.com/GoogleCloudPlatform/gsutil/issues/961#issuecomment-604648510

Tracking this down, this error comes from a change in Python 3.8 in the multiprocessing library:

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Spawn is being run for those using MacOs and Python 3.8+ by default since nothing is explicitly set either through get_context or set_start_method.

[allekmott]

I've created #242 to address this.

[christopherowen]

This is a much better implementation than my hack to change the processing context to fork.

Also, there's another PR lying around that changes the multiprocessing to use a pool. It may be interesting too.

[aboul3la]

Thanks guys for bringing this to my attention, I just merged #242 that should fix this issue.