putty is not flake8 v3 compatible

[jayvdb]

flake8_putty is broken for pywikibot, likely due to a change in either flake8 or pep8/pycodestyle.

18:31:33 + tox -v
18:31:33 using tox.ini: /home/jenkins/workspace/tox-jessie/tox.ini
18:31:33 using tox-1.9.2 from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc
18:31:33 flake8 create: /home/jenkins/workspace/tox-jessie/.tox/flake8
18:31:33   /home/jenkins/workspace/tox-jessie/.tox$ /usr/bin/python -m virtualenv --setuptools --python /usr/bin/python2.7 flake8 >/home/jenkins/workspace/tox-jessie/.tox/flake8/log/flake8-0.log
18:31:36 flake8 installdeps: flake8, pyflakes >= 1.1, hacking, flake8-docstrings>=0.2.6, flake8-putty>=0.3.2, flake8-coding, flake8-future-import, flake8-string-format, flake8-import-order, flake8-tuple>=0.2.8, flake8-print>=2.0.1, git+https://github.com/jayvdb/flake8-mock@use-ascii-readme
18:31:36   /home/jenkins/workspace/tox-jessie$ /home/jenkins/workspace/tox-jessie/.tox/flake8/bin/pip install --process-dependency-links --pre flake8 pyflakes >= 1.1 hacking flake8-docstrings>=0.2.6 flake8-putty>=0.3.2 flake8-coding flake8-future-import flake8-string-format flake8-import-order flake8-tuple>=0.2.8 flake8-print>=2.0.1 git+https://github.com/jayvdb/flake8-mock@use-ascii-readme >/home/jenkins/workspace/tox-jessie/.tox/flake8/log/flake8-1.log
18:31:40 flake8 develop-inst: /home/jenkins/workspace/tox-jessie
18:31:40   /home/jenkins/workspace/tox-jessie$ /home/jenkins/workspace/tox-jessie/.tox/flake8/bin/pip install --process-dependency-links --pre -e /home/jenkins/workspace/tox-jessie >/home/jenkins/workspace/tox-jessie/.tox/flake8/log/flake8-2.log
18:31:42 flake8 runtests: PYTHONHASHSEED='646594070'
18:31:42   /home/jenkins/workspace/tox-jessie$ /home/jenkins/workspace/tox-jessie/.tox/flake8/bin/flake8 --version 
18:31:42 Traceback (most recent call last):
18:31:42   File ".tox/flake8/bin/flake8", line 11, in <module>
18:31:42     sys.exit(main())
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/main/cli.py", line 16, in main
18:31:42     app.run(argv)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/main/application.py", line 292, in run
18:31:42     self._run(argv)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/main/application.py", line 278, in _run
18:31:42     self.initialize(argv)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/main/application.py", line 269, in initialize
18:31:42     self.register_plugin_options()
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/main/application.py", line 149, in register_plugin_options
18:31:42     self.check_plugins.register_options(self.option_manager)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/plugins/manager.py", line 378, in register_options
18:31:42     list(self.manager.map(call_register_options))
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/plugins/manager.py", line 252, in map
18:31:42     yield func(self.plugins[name], *args, **kwargs)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/plugins/manager.py", line 348, in generated_function
18:31:42     return method(optmanager, *args, **kwargs)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8/plugins/manager.py", line 198, in register_options
18:31:42     add_options(optmanager)
18:31:42   File "/home/jenkins/workspace/tox-jessie/.tox/flake8/local/lib/python2.7/site-packages/flake8_putty/extension.py", line 118, in add_options
18:31:42     parser.config_options.append('putty-select')
18:31:42 AttributeError: 'OptionManager' object has no attribute 'config_options'
18:31:42 ERROR: InvocationError: '/home/jenkins/workspace/tox-jessie/.tox/flake8/bin/flake8 --version'

investigating...

[jayvdb]

After fixing all the other problems that have emerged since the last build, I assume that this is because pywikibot's tox.ini is installing flake8 v3. Need to test that theory yet.

[The-Compiler]

flake8 documentation about the change: http://flake8.pycqa.org/en/latest/plugin-development/cross-compatibility.html#option-handling-on-flake8-2-and-3

Related fixes in pep8-naming:

https://github.com/PyCQA/pep8-naming/commit/b4c3ebd4884bac27fae56aa45c6f9e7c5b5ea5a0 https://github.com/PyCQA/pep8-naming/commit/dd623fcd76db58922a511578ff5d14ce741f9636

[The-Compiler]

Also, let me point out the log above includes --pre when calling pip, so I think your theory is correct.

[jayvdb]

Yea, the fix for pywikibot was to remove the --pre , and is merged.

It was easy to bypass the flake8 v3 option handling problem in the opening bug report, but as flake8 v3 is not using pep8 for the StyleGuide, file parsing and other structural classes, getting a hook inside of flake8 will need a lot of work.

However I am still debating in #15 whether this flake8 is needed in a flake8 v3 world.

[jayvdb]

For now I have updated setup.py in master to indicate it is incompatible with v3. I'll push this out to pypi as 0.3.3 , along with some other minor fixes, in a few days time, unless v3 comes out of beta sooner.

[jayvdb]

Apparently 19bc89a1cd is not enough to prevent flake8-putty being used with flake8-v3, as seen at https://phabricator.wikimedia.org/T141257 Possibly caused by older pip/setuptools

(the workaround is to pin flake8<3 in every single project .... huge stupid sigh)

[jayvdb]

And flake8 v3 is not compatible with Python 2.6

$ git grep OrderedDict
src/flake8/utils.py:        parameters = collections.OrderedDict([
src/flake8/utils.py:        parameters = collections.OrderedDict([

[omriharel]

Would love to see this extension working with flake8 v3, I need it to filter specific "imported but not used" errors (based on the imported module) in my project. I would use v2 but that renders my own custom checker plugin useless.

[jayvdb]

Hi @omriharel , what is your own custom checker plugin, and why it is useless on v2?

See my response at https://github.com/jayvdb/flake8-putty/issues/15#issuecomment-240432557

Another critical bug just reported: https://gitlab.com/pycqa/flake8/issues/214

And many more by @asottile about the unusable state of the legacy API, without a replacement API.

flake8-putty fiddles inside the stack frames of flake8 and pycodestyle, so it is not sensible making it work when flake8 v3 internals are very likely to change. But if you really, really need it, I can at least investigate how much of -putty can be achieved without fiddling inside flake8/pycodestyle.

[omriharel]

Hi @jayvdb, I think it may be useless on v2 due to my own fault: I wasn't able to understand the plugin documentation very well (and assumed v3 was stable for some reason), so I just delved into its code and tried to figure out how pyflakes and pep8 themselves are plugged into flake8.

My plugin is essentially a collection of checkers added on top of the default plugins (each in their own top level function being registered as a setuptools entry point, similar to the above examples), looking for team and project-specific infractions (and common gotchas) in our medium-sized codebase.

I wasn't able to figure out how to properly make it work with v2, and between this and my particular filtering need, I'd rather just disable the unused import checks until I'm able to use flake8-putty.

This isn't something I really, really need, but would enjoy seeing if and when it becomes possible - thank you for the kind offer, however :)

[lemcheyWhite]

Hi, @jayvdb do you have any plans to continue working on this?

[jayvdb]

See https://github.com/jayvdb/flake8-putty/issues/15#issuecomment-310802041

The v2 compatible version of putty is a hack, fiddling with the stack to bypass some of flake8 in order for this plugin to work. This is reasonable to maintain because v2 is very stable, and it is unlikely that upstream will change its internals, so the hack wont break.

If I provide v3 support using a hack, it is quite likely that it will break, and I am then committed to try and find a solution.

Also, flake8 v3 was a large rewrite which removed the section of code that I was using to get access to flake8 internals, so it will be much harder to reconstruct the hack on flake8 v3, and I'd prefer to not build a hacky version for flake8 v3 anyway.

I hope that flake8 will eventually allow plugins to achieve the same result using a stable API.

[jayvdb]

Also, fwiw, the other very important flake8 plugin for me is hacking, and they also are pinned at flake8 v2. So all my projects that use flake8 are also stuck on flake8 v2 until that is resolved.

[snoack]

I published a similar flake8 extension, which is compatible with flake8 3, it's flake8-per-file-ignores.

[jayvdb]

Great @snoack , I would be happy to accept a README update here linking to your project.

[snoack]

Cool. There you go: #21