Incorrect inferred types and list not iterable issues/questions

[Nekodann]

After one of the updates (which exactly I, unfortunately, don't remember) Pylance started working very weird. A lot of errors appeared that haven't been there before. For example, I have the following test code with looping through the list.

test_list = [1, 2, 3]

for i in test_list:
    print(i)

Which certainly works and is fine, but Pylance is showing an error saying that:

"list[int]" is not iterable "next" method not defined on type "Iterator" Pylance[reportGeneralTypeIssues] (variable) test_list: list[int]

One other issue is that for more difficult types Pylance is not able to infer the variable type even though typing is used:

Along some other, similar nuances. My question is whether this is the desired behavior and should be this way? Or is it some misunderstanding on my part? Any answer would be appreciated.

Versions: Python 3.9.18 VS Code 1.82.2 on Ubuntu 22.04.3 LTS, installed using snap Python extension v2023.16.0 Pylance v2023.9.20

[rchiodo]

Thanks for the issue. I can't reproduce but that's not surprising as that error seems way off.

My only guess is that typeshed files are not present or are being overwritten by something else.

For example, if I change your example to have list[int] and then goto def, I get this as the definition of list.

Can you include your log as described here?

https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue

That should show us where your typeshed files are coming from.

[Nekodann]

Managed to get the log, but this is the company repository, so I am not sure how much I can share. I actually tried to reproduce this by creating a new environment and I did not have this issue, everything worked as expected with Pylance.

How much of the log do you need? Or is it not a problem to include the entirety of it?

Edit: I managed to reproduce the issue on a fresh repository. The issue lies with the environment. Simply creating a new folder and copying the venv folder and selecting the interpreter to use the Python from the virtual env is enough to break the extension. I am using venv.

I am attaching the log as a file. Python Language Server.log

[rchiodo]

What happens when you go to definition on list[int]? The log doesn't show an different typeshed being used, so my initial guess seems wrong.

[rchiodo]

Actually, from the error message, I'm thinking it's the definition of Iterable. So goto def on something like this:

from typing import Iterable

x: Iterable[int] = [1, 2, 3]

Goto def for iterable for me ends up here:

[erictraut]

That's the definition (i.e. the implementation). What happens if you "Go To Declaration" instead? It should take you to a stub file (typing.pyi).

[Nekodann]

I get an error on int:

Both the definition and the declaration goes to here:

[rchiodo]

Can you clear the log, then do goto declaration on Iterator and send us the log for that?

Thanks.

[Nekodann]

Of course.

Python Language Server 2.log

[rchiodo]

Could you upload the output of pip list? I don't see it even looking for typing.pyi, not sure why. Maybe I can reproduce if I have the same environment.

For me just opening the file with the code above causes typing.pyi to be parsed.

2023-09-28 16:19:08.933 [info] (10340) [BG(2)]   binding: C:\Users\rchiodo\source\testing\Test_Typing\test.py (0ms)
2023-09-28 16:19:08.933 [info] (10340) [BG(2)]   checking: C:\Users\rchiodo\source\testing\Test_Typing\test.py ...
2023-09-28 16:19:08.933 [info] (10340) [BG(2)]     parsing: C:\Users\rchiodo\.vscode-insiders\extensions\ms-python.vscode-pylance-2023.9.31\dist\typeshed-fallback\stdlib\typing.pyi [fs read 7ms] (48ms)
2023-09-28 16:19:08.934 [info] (10340) [BG(2)]     binding: C:\Users\rchiodo\.vscode-insiders\extensions\ms-python.vscode-pylance-2023.9.31\dist\typeshed-fallback\stdlib\typing.pyi (10ms)

For you at the same location, it just jumps to the real implementation instead of the type stubs:

2023-09-28 11:50:17.296 [info] (80049) [BG(1)]   checking: /home/daniel/Projects/reproduce-project/test.py ...
2023-09-28 11:50:17.296 [info] (80049) [BG(1)]     parsing: /home/daniel/Projects/reproduce-project/ta_env/lib/python3.9/site-packages/typing.py [fs read 0ms] (76ms)

[Nekodann]

This is my pip list:

Package                                 Version
--------------------------------------- -----------
adal                                    1.2.6
aenum                                   3.1.15
aiohttp                                 3.7.4.post0
ansicon                                 1.89.0
appdirs                                 1.4.4
asgiref                                 3.3.1
async-timeout                           3.0.1
attrs                                   20.3.0
aws-xray-sdk                            2.8.0
Babel                                   2.9.0
backports.csv                           1.0.7
beautifulsoup4                          4.9.3
blessed                                 1.18.0
boto                                    2.49.0
boto3                                   1.17.38
botocore                                1.20.38
businesstime                            0.3.0
cached-property                         1.5.2
cachetools                              4.2.1
caldav                                  0.8.0
cement                                  3.0.4
certifi                                 2020.12.5
cffi                                    1.14.5
chardet                                 4.0.0
click                                   7.1.2
colorama                                0.4.4
colorlog                                4.8.0
confidential                            2.3.1
coreapi                                 2.3.3
coreschema                              0.0.4
croniter                                1.4.1
cryptography                            3.4.7
cssselect2                              0.4.1
DateTimeRange                           1.2.0
decorator                               5.1.1
defusedxml                              0.7.1
Deprecated                              1.2.14
diff-match-patch                        20200713
Django                                  3.1.13
django-3-jet                            1.0.8
django-admin-inline-paginator           0.3.0
django-admin-json-editor                0.2.3
django-admin-list-filter-dropdown       1.0.3
django-after-response                   0.2.2
django-allauth                          0.44.0
django-angular                          2.3
django-appconf                          1.0.4
django-auth-adfs                        1.7.0
django-colorfield                       0.4.1
django-common-helpers                   0.9.2
django-cors-headers                     3.7.0
django-cron                             0.5.1
django-csp                              3.7
django-dbbackup                         3.3.0
django-debug-toolbar                    3.2
django-debug-toolbar-request-history    0.1.3
django-environ                          0.4.5
django-extensions                       3.1.1
django-filter                           2.4.0
django-groupadmin-users                 0.3.4
django-guardian                         2.3.0
django-hosts                            4.0
django-htmlmin                          0.11.0
django-ical                             1.7.1
django-import-export                    2.5.0
django-jsonschema-form                  1.0.3
django-macaddress                       1.7.0
django-nose                             1.4.7
django-oauth-toolkit                    1.5.0
django-okta-auth                        0.8.0
django-phonenumber-field                5.0.0
django-recurrence                       1.10.3
django-redis                            5.2.0
django-rest-framework-datatables-filter 0.0.2
django-revproxy                         0.10.0
django-rq                               2.7.0
django-sslserver                        0.22
django-storages                         1.11.1
django-tqdm                             1.0.0
djangorestframework                     3.12.4
djangorestframework-csv                 2.1.0
djangorestframework-datatables          0.6.0
djangorestframework-simplejwt           5.2.0
djangorestframework-xml                 2.0.0
djangular                               0.2.7
dnspython                               2.1.0
docker-py                               1.10.6
docker-pycreds                          0.4.0
dockerpty                               0.4.1
docopt                                  0.6.2
docutils                                0.16
drf-nested-routers                      0.93.3
drf-writable-nested                     0.6.2
ecdsa                                   0.14
elementpath                             4.1.5
et-xmlfile                              1.0.1
factory-boy                             3.2.1
Faker                                   19.6.2
flatdict                                4.0.1
freezegun                               1.1.0
future                                  0.18.2
gevent                                  21.1.2
google-api-core                         1.26.2
google-api-python-client                2.19.0
google-auth                             1.28.0
google-auth-httplib2                    0.1.0
google-auth-oauthlib                    0.4.3
googleapis-common-protos                1.53.0
graypy                                  2.1.0
greenlet                                1.0.0
grequests                               0.6.0
html5lib                                1.1
httplib2                                0.19.0
humanize                                3.3.0
hurry.filesize                          0.9
icalendar                               4.0.7
idna                                    2.8
idna-ssl                                1.1.0
importlib-metadata                      3.7.3
importlib-resources                     5.1.2
isodate                                 0.6.0
itypes                                  1.2.0
jdcal                                   1.4.1
Jinja2                                  2.11.3
jinxed                                  1.0.1
jmespath                                0.10.0
jsonpickle                              2.0.0
jwcrypto                                1.5.0
lxml                                    4.9.2
MarkupPy                                1.14
MarkupSafe                              1.1.1
mbstrdecoder                            1.1.3
mock                                    4.0.3
msal                                    1.21.0
multidict                               5.1.0
netaddr                                 0.8.0
nose                                    1.3.4
ntlm-auth                               1.5.0
oauthlib                                3.1.0
odfpy                                   1.4.1
okta                                    2.9.2
olefile                                 0.46
openpyxl                                3.0.7
packaging                               20.9
pathspec                                0.8.1
pbr                                     5.5.1
phonenumbers                            8.12.20
Pillow                                  8.3.1
pip                                     21.2.1
protobuf                                3.15.6
psycopg2                                2.9.5
psycopg2-binary                         2.9.5
py                                      1.11.0
pyasn1                                  0.4.8
pyasn1-modules                          0.2.8
pycparser                               2.20
pycryptodomex                           3.19.0
pydash                                  4.9.3
pyfcm                                   1.5.1
Pygments                                2.8.1
PyJWT                                   1.7.1
pyOpenSSL                               21.0.0
pyparsing                               2.4.7
pysaml2                                 7.3.1
python-dateutil                         2.8.1
python-jose                             3.3.0
python3-openid                          3.2.0
pytz                                    2021.1
PyYAML                                  5.4.1
qrcode                                  6.1
redis                                   4.1.4
reportlab                               3.5.66
requests                                2.25.1
requests-ntlm                           1.1.0
requests-oauthlib                       1.3.0
rest-condition                          1.0.3
retry                                   0.9.2
rq                                      1.12.0
rq-scheduler                            0.11.0
rsa                                     4.7.2
s3transfer                              0.3.6
sentry-sdk                              1.0.0
setuptools                              54.2.0
six                                     1.15.0
soupsieve                               2.2.1
sqlparse                                0.4.1
svglib                                  1.0.1
tablib                                  3.0.0
texttable                               1.6.3
tinycss2                                1.1.0
tqdm                                    4.66.1
typepy                                  1.3.1
typing                                  3.7.4.3
typing-extensions                       3.7.4.3
tzlocal                                 2.1
unicodecsv                              0.14.1
uritemplate                             3.0.1
urllib3                                 1.26.4
vobject                                 0.9.6.1
wcwidth                                 0.2.5
webencodings                            0.5.1
websocket-client                        0.58.0
whatever                                0.6
wheel                                   0.36.2
wrapt                                   1.12.1
xlrd                                    2.0.1
xlwt                                    1.3.0
xmlschema                               2.5.0
xmltodict                               0.13.0
yarl                                    1.6.3
zipp                                    3.4.1
zope.event                              4.5.0
zope.interface                          5.3.0

[rchiodo]

No such luck. Still can't reproduce the problem. Does this file exist?

extensions/ms-python.vscode-pylance-2023.9.31/dist/typeshed-fallback/stdlib/typing.pyi

[Nekodann]

No, I am not able to find it. Could you point me where I should be able to find it? Also, would it help for you to have the entire venv folder?

[rchiodo]

Oh sorry, should have included more of the path there for the typing.pyi.

From your logs, I believe that your extensions folder is here:

/home/daniel/.vscode/extensions/ms-python.vscode-pylance-2023.9.30

So the path to that file should be this:

/home/daniel/.vscode/extensions/ms-python.vscode-pylance-2023.9.30/dist/typeshed-fallback/stdlib/typing.pyi

Can you check that the file exists and that it has content related to Iterable?

[rchiodo]

I don't think the entire venv folder would be transferable. I installed everything you had listed so it should be really close to the same.

[Nekodann]

Found the file in exactly the place you provided, thank you. In it there is a declaration of Iterable:

@runtime_checkable
class Iterable(Protocol[_T_co]):
    @abstractmethod
    def __iter__(self) -> Iterator[_T_co]: ...

And of course it's used in various places in typying.pyi

[rchiodo]

Hmm, not sure how it doesn't find it then. If you search in the venv for a typings.pyi, is it found anywhere?

[Nekodann]

This is the output for find venv/ -type f -name typing.pyi:

./ta_env/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/2/typing.pyi
./ta_env/lib/python3.9/site-packages/jedi/third_party/typeshed/stdlib/3/typing.pyi

So I assume this is not the correct typing file needed?

[rchiodo]

I'm guessing it would work if you removed jedi from your list of packages. Which is weird because jedi wasn't in the pip list.

[rchiodo]

Oh wait that's your global python. That explains why it wasn't in the other pip list.

No that's the venv. Weird.

[Nekodann]

Yeah just double checked and indeed it is not in pip list and it should not be there, as is the case

[rchiodo]

Closing old issue. I believe Jedi was the culprit here.