WPS335 rejects list-of-lists comprehensions

[astynax]

What's wrong

I'm trying to produce a list of lists but getting a false-positive warning:

WPS335 Found incorrect `for` loop iter type
 ([x] for x in (1, 2, 3))
 ^

Here is a code

[ l for l in
  ([x] for x in (1, 2, 3))
]

How it should be

It should be possible to produce the list of lists.

Flake8 version and plugins

{
  "dependencies": [],
  "platform": {
    "python_implementation": "CPython",
    "python_version": "3.8.8",
    "system": "Linux"
  },
  "plugins": [
    {
      "is_local": false,
      "plugin": "flake8-bandit",
      "version": "2.1.2"
    },
    {
      "is_local": false,
      "plugin": "flake8-broken-line",
      "version": "0.3.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-bugbear",
      "version": "20.11.1"
    },
    {
      "is_local": false,
      "plugin": "flake8-comprehensions",
      "version": "3.5.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-darglint",
      "version": "1.8.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-debugger",
      "version": "4.0.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-docstrings",
      "version": "1.6.0, pydocstyle: 6.0.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-eradicate",
      "version": "1.0.0"
    },
    {
      "is_local": false,
      "plugin": "flake8-string-format",
      "version": "0.3.0"
    },
    {
      "is_local": false,
      "plugin": "flake8_commas",
      "version": "2.0.0"
    },
    {
      "is_local": false,
      "plugin": "flake8_isort",
      "version": "4.0.0"
    },
    {
      "is_local": false,
      "plugin": "flake8_quotes",
      "version": "3.2.0"
    },
    {
      "is_local": false,
      "plugin": "mccabe",
      "version": "0.6.1"
    },
    {
      "is_local": false,
      "plugin": "naming",
      "version": "0.11.1"
    },
    {
      "is_local": false,
      "plugin": "pycodestyle",
      "version": "2.7.0"
    },
    {
      "is_local": false,
      "plugin": "pyflakes",
      "version": "2.3.1"
    },
    {
      "is_local": false,
      "plugin": "rst-docstrings",
      "version": "0.0.14"
    },
    {
      "is_local": false,
      "plugin": "wemake_python_styleguide",
      "version": "0.15.2"
    }
  ],
  "version": "3.9.2"
}

pip information

pip 20.2.3 from /home/astynax/Projects/hexlet/hexlet-exercise-kit/exercises/course-python-declarative-programming/python_declarative_programming_list_comprehensions_exercise/.venv/lib/python3.8/site-packages/pip (python 3.8) astor==0.8.1 attrs==21.2.0 bandit==1.7.0 darglint==1.8.0 docutils==0.17.1 eradicate==2.0.0 flake8==3.9.2 flake8-bandit==2.1.2 flake8-broken-line==0.3.0 flake8-bugbear==20.11.1 flake8-commas==2.0.0 flake8-comprehensions==3.5.0 flake8-debugger==4.0.0 flake8-docstrings==1.6.0 flake8-eradicate==1.0.0 flake8-isort==4.0.0 flake8-polyfill==1.0.2 flake8-quotes==3.2.0 flake8-rst-docstrings==0.0.14 flake8-string-format==0.3.0 gitdb==4.0.7 GitPython==3.1.14 isort==5.8.0 mccabe==0.6.1 pbr==5.6.0 pep8-naming==0.11.1 pycodestyle==2.7.0 pydocstyle==6.0.0 pyflakes==2.3.1 Pygments==2.9.0 PyYAML==5.4.1 restructuredtext-lint==1.3.2 six==1.16.0 smmap==4.0.0 snowballstemmer==2.1.0 stevedore==3.3.0 testfixtures==6.17.1 typing-extensions==3.10.0.0 wemake-python-styleguide==0.15.2

OS information

Ubuntu Linux 16.04

[sobolevn]

Yes, this should be allowed for comprehensions.

[ujju20]

Hello, I am a beginner in the Open source world and I found this issue to be good first so I would like to take this issue. So please guide me.

[sobolevn]

@ujju20 let's focus on https://github.com/wemake-services/wemake-python-styleguide/issues/1825 🙂

[sobolevn]

I am looking through the bug reports and now I thing that this is one is invaldid. Your initial example should be written as:

>>> list([x] for x in (1, 2, 3))
[[1], [2], [3]]

Maybe you have other examples in mind?

[astynax]

O, rly? This is just a minimal example! And why it's invalid? One may want to iterate from the subcomprehension! Here is a real example if you want:

def non_empty_truths(list_of_lists):
    return [
        truths for truths in
        ([elem for elem in one_list if elem]  # noqa: WPS335
         for one_list in list_of_lists
         )
        if truths
    ]

And here should be the lists! Not tuples, not genexps, the lists!

[Sxderp]

Doesn't this create an extra generator?

# Same a above but rewritten on less lines
[
    truths
    for truths in ([elem for elem in one_list if elem] for one_list in list_of_lists)
    if truths
]

In many contexts generators are /slow/. In some they're only a little slower than a list, but in others (like when you need length) they're way slower. I'd argue that replacing the generator with a list is also valid.

[
    truths
    for truths in [[elem for elem in one_list if elem] for one_list in list_of_lists]
    if truths
]

You /can/ break this out into two steps and the linter is happy with it. I don't know. seems a bit excessive.

step_one = [[elem for elem in one_list if elem] for one_list in list_of_lists]
[truths for truths in step_one if truths]

[surya-kodeti]

Hi there, I'm a newbie searching for an easy-to-solve issue this seems to be a good start for me. If this is still open, please guide me on this

[kameshwarasekar]

I support answer by @Sxderp. You dont need an iterator in this case. In the initial example, the example you provided can be simplifies as: list_of_list = [[x] for x in (1, 2, 3)]

[CodeCraftsman481]

Hi, I am new to contributing, Can I take this task up?

[Sxderp]

I want to point out that I was not in favor of keeping WPS335 as-is. I do believe there are valid use cases. Here is another one that we just came across, turning a comma-separated list of arguments in the format arg1=val1 into a dict. Since we want to perform additional type-casting the extra comprehension is necessary.

from contextlib import suppress

def _num_or_str(sval):
    try:
        return int(sval)
    except ValueError:
        return str(sval)

argstr = 'day=2,month=December'
argdict = { 
    kword: _num_or_str(kword_arg)
    for kword, kword_arg in [arg.split('=') for arg in argstr.split(',')]
}
print(argdict)