search

ayivima / flake8-lineleak

Keep your scripts short, do more modular programming...Else get warned :)
MIT License
2 stars 2 forks source link

PicklingError #1

Open Dreamsorcerer opened 5 years ago

Dreamsorcerer commented 5 years ago

If I don't set jobs = 1 in the flake8 config, then I get this exception:

Traceback (most recent call last):
  File "/usr/local/bin/flake8", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/flake8/main/cli.py", line 18, in main
    app.run(argv)
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 394, in run
    self._run(argv)
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 382, in _run
    self.run_checks()
  File "/usr/local/lib/python3.7/site-packages/flake8/main/application.py", line 301, in run_checks
    self.file_checker_manager.run()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 328, in run
    self.run_parallel()
  File "/usr/local/lib/python3.7/site-packages/flake8/checker.py", line 292, in run_parallel
    for ret in pool_map:
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 354, in <genexpr>
    return (item for chunk in result for item in chunk)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/pool.py", line 431, in _handle_tasks
    put(task)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'flake8-lineleak.Screener'>: import of module 'flake8-lineleak' failed
        2.66 real         2.08 user         0.47 sys

Possibly related to: https://gitlab.com/pycqa/flake8/issues/164 To make it compatible with flake8 version 3+, it may need a similar update as this plugin: https://github.com/JetBrains/teamcity-messages/pull/106/files

ayivima commented 5 years ago

Glad to have this feedback @Dreamsorcerer. I checked out the links and it seems this is possibly a problem with flake8's jobs. For further clarification, do you mean lineleak works if jobs is set to 1? Or the exception is just silenced?

Dreamsorcerer commented 5 years ago

Pretty sure it is running correctly when using 1 job (it just takes flake8 5 times longer to run). If you look at the linked issue, it seems to be something that occurs only when the multiprocessing module is used (as can be spotted in the traceback).

ayivima commented 5 years ago

That's good feedback...I am looking into it...

yahkun commented 4 years ago

Any progress?