Josverl / micropython-stubs

Stubs of most MicroPython ports, boards and versions to make writing code that much simpler.
https://micropython-stubs.readthedocs.io
MIT License
133 stars 21 forks source link

`namedtuple` Pylance error `Variable not allowed in type expression` #731

Closed North101 closed 6 months ago

North101 commented 6 months ago

I have the following code and i'm not sure how best to resolve the error:

WifiConfig = namedtuple('WifiConfig', ('ssid', 'password'))

def read_wifi_config() -> WifiConfig | None:
  try:
    with open(config.WIFI_FILE, 'r') as f:
      data = json.load(f)
      f.close()

    return WifiConfig(
        ssid=data['ssid'],
        password=data.get('password'),
    )
  except:
    return None
Screenshot 2023-12-14 at 15 21 55 Screenshot 2023-12-14 at 15 25 23

Is this a problem with the stubs or a limitation of trying to use Pylance with micropython?

Josverl commented 6 months ago

Hi , thanks for reporting.

I am already have a todo on another issue with NamedTuples. A QA test has been created : https://github.com/Josverl/micropython-stubs/blob/main/tests/quality_tests/feat_stdlib/check_collections/check_namedtuple.py but I've not had time to solve this yet.

Im not sure yet where the problem lies:

The snippet checks out on the pyright playground

Does it run OK in Micropython ? on what port / board ?

North101 commented 6 months ago

It runs just fine on the device, its only showing linting errors when the stubs are installed.

I'm using these stubs: micropython-rp2-rpi_pico_w-stubs

Josverl commented 6 months ago

I think I found the cause of the problem

In the generation of the stubs I cast the return to typing_extensions.NamedTuple rather than stdlib.collections.namedtuple I assumed the were equal, but apparently they are not

# .../collections.pyi
from stdlib.collections import namedtuple as stdlib_namedtuple # <--- Add this
# from typing_extensions import NamedTuple as stdlib_NamedTuple # not this

def namedtuple(name, fields) -> stdlib_namedtuple: # <-- change this 
   ...

I need some time to integrate this and generate new stubs , but for now you could manually update the stub.

as I side note : I think you should add a guard clause even with the above fix,

...
    if not isinstance(data, dict):
        return None
    return WifiConfig(
        ssid=data['ssid'],
        password=data.get('password'),
    )
  except:
    return None

to prevent : image

North101 commented 6 months ago
# .../collections.pyi
from stdlib.collections import namedtuple as stdlib_namedtuple # <--- Add this
# from typing_extensions import NamedTuple as stdlib_NamedTuple # not this

def namedtuple(name, fields) -> stdlib_namedtuple: # <-- change this 
   ...

This worked for me. Thanks!

as I side note : I think you should add a guard clause even with the above fix,

...
    if not isinstance(data, dict):
        return None
    return WifiConfig(
        ssid=data['ssid'],
        password=data.get('password'),
    )
  except:
    return None

I'm not getting that linting error, but good point. Thanks

Josverl commented 6 months ago

I could not get consistent results - I may need to update micropython-stdlib to get consistent results across the board Happy the workaround works on your system - Which Python version / OS are you running ?

North101 commented 6 months ago

Python 3.11 MacOS

North101 commented 6 months ago

After some more testing, it seems to work fine unmodified with venv, but if I install via --target ./typings --no-user, thats when I get the error.

Clicking on the collections or namedtuple import and selecting Go to Definition or Go to Declaration takes me to the source code of my systems python instead of the one definition in .venv/lib/python3.11/site-packages. The same happens with asyncio and json, but not with sys or time.

North101 commented 6 months ago

Ok, after experimenting more I think your solution didn't work, instead I think what was happening was it was defaulting back to my installed python types

Josverl commented 6 months ago

@North101 , I came to a similar conclusion. My current understanding is that the type checkers have some "special case handling" to make type-checking work for namedtuples. : https://github.com/python/typeshed/blob/ba7bd9f98ad9bba957dfb4175eac5448f6273575/stdlib/collections/__init__.pyi#L38

I think that means that namedtuples typechecking works only for that exact module: stdlib_stubs.collections.nametuple and not for the level up where the micropython definition lives.

I'm currently working to test if i can automate updates to the typeshed stdlib stubs, and if that solves more than it may break.

Josverl commented 6 months ago

@North101 Santa delivered a present-stub for you :-)

image

(same with ucollections)

Feel free to re-open if you find any issues

North101 commented 6 months ago

Thanks! It works