Bitwise operator between classes is not recognized correctly

[sevdog]

Environment data

• Language Server version: v2023.1.30
• OS and version: Ubuntu 22.04 LTS
• Python version: CPython 3.9

Code Snippet

# djangorestframework==3.14.0
from rest_framework.permissions import BasePermission

class PermissionA(BasePermission):
   ...

class PermissionB(BasePermission):
   ...

permission = PermissionA | PermissionB

Repro Steps

1. setup a project in python 3.9 which uses django-rest-framework (>=3.9)
2. create at least 2 permission classes
3. use a bitwise operator to combine those classes

   Provided they inherit from rest_framework.permissions.BasePermission, permissions can be composed using standard Python bitwise operators

Expected behavior

No hints should be raised.

Actual behavior

Pylance outputs the following hit at error level:

Alternative syntax for unions requires Python 3.10 or newer

Logs

Unable to obtains detailed logs, in settings.json the option is faded and displays the following hint:

This setting cannot be applied in this workspace. It will be applied when you open the containing workspace folder directly.

[erictraut]

I'm not able to repro the problem you're seeing with the above code and repro steps. I do see the error prior to installing the djangoresframework library in the Python environment, but once the package is installed, the error goes away. The BasePermission class derives from a metaclass called BasePermissionMetaclass which implements bitwise operations including __or__.

I suspect that your problem is related to either not selecting the correct Python environment or not properly installing the djangoresframework library into that environment. Please verify that your configuration is correct.

[sevdog]

@erictraut I have reproduced a minimal configuration for this issue and it is not fault of any other configurations:

To reproduce:

1. Install python 3.9
2. Create a virtualenv for python 3.9 (ie virtualenv -p python3.9 .venv)
3. Activate virtualenv (source .venv/bin/activate)
4. Install DRF (pip install djangorestframework)

With jedi I did not have this kind of problem.

[rchiodo]

Is this output the answer to the question?

Alternative syntax for unions requires Python 3.10 or newer

This syntax isn't supported in Python 3.9.

[rchiodo]

With python 3.9, I believe you need to import annotations?

https://github.com/microsoft/pylance-release/issues/542#issuecomment-730415446

I haven't tried to repro with 3.9 myself though.

[erictraut]

@rchiodo, the classes in question derive from a metaclass that implements __or__, so the expression PermissionA | PermissionB invokes that magic method. This is not a union operator, even though it uses the same syntax.

I'm still not able to repro the problem as described. @rchiodo, are you able to?

[rchiodo]

No I can't repro. It works fine for me:

[sevdog]

The problem is still there in my environment. I got pylance logs working (for some reason under Ubuntu 22 it was not working when I filed this issue) here they are: pylance.log.

For what I can figure it is not indexing/parsing some folders in the virtualenv:

$ grep .venv pylance.log 
[Info  - 10:49:15] (180797) Setting pythonPath for service "<default>": "/tmp/testpylance/.venv/bin/python"
[Info  - 10:49:15] (180797)   /tmp/testpylance/.venv/lib/python3.9/site-packages
[Info  - 10:49:15] (180797) Setting pythonPath for service "testpylance": "/tmp/testpylance/.venv/bin/python"
[Info  - 10:49:15] (180797)   /tmp/testpylance/.venv/lib/python3.9/site-packages
/tmp/testpylance/.venv/lib/python3.9/site-packages
[Info  - 10:49:15] (180797) Setting pythonPath for service "<default>": "/tmp/testpylance/.venv/bin/python"
[Info  - 10:49:16] (180797)   /tmp/testpylance/.venv/lib/python3.9/site-packages
(180797) [IDX(3)]     parsing: /tmp/testpylance/.venv/lib/python3.9/site-packages/asgiref/__init__.py [fs read 0ms] (0ms)
(180797) [IDX(3)]     binding: /tmp/testpylance/.venv/lib/python3.9/site-packages/asgiref/__init__.py (0ms)
(180797) [IDX(3)]     indexing: /tmp/testpylance/.venv/lib/python3.9/site-packages/asgiref/__init__.py [found 1] (1ms)
(180797) [IDX(3)]     parsing: /tmp/testpylance/.venv/lib/python3.9/site-packages/pip/__init__.py [fs read 0ms] (1ms)
(180797) [IDX(3)]     binding: /tmp/testpylance/.venv/lib/python3.9/site-packages/pip/__init__.py (0ms)
(180797) [IDX(3)]     indexing: /tmp/testpylance/.venv/lib/python3.9/site-packages/pip/__init__.py [found 2] (0ms)
(180797) [IDX(3)]     parsing: /tmp/testpylance/.venv/lib/python3.9/site-packages/sqlparse/__init__.py [fs read 0ms] (6ms)
(180797) [IDX(3)]     binding: /tmp/testpylance/.venv/lib/python3.9/site-packages/sqlparse/__init__.py (2ms)
(180797) [IDX(3)]     indexing: /tmp/testpylance/.venv/lib/python3.9/site-packages/sqlparse/__init__.py [found 0] (1ms)

These are the packages in /tmp/testpylance/.venv/lib/python3.9/site-packages folder:

$ ls .venv/lib/python3.9/site-packages 
asgiref                   Django-4.1.6.dist-info                pkg_resources            setuptools                    _virtualenv.pth
asgiref-3.6.0.dist-info   djangorestframework-3.14.0.dist-info  __pycache__              setuptools-65.6.3.dist-info   _virtualenv.py
_distutils_hack           pip                                   pytz                     setuptools-65.6.3.virtualenv  wheel
distutils-precedence.pth  pip-22.3.1.dist-info                  pytz-2022.7.1.dist-info  sqlparse                      wheel-0.38.4.dist-info
django                    pip-22.3.1.virtualenv                 rest_framework           sqlparse-0.4.3.dist-info      wheel-0.38.4.virtualenv

From the logs it is not parsing:

• rest_framework
• pytz
• django (but it is using a bundled stubs)
• setuptools
• wheel

Is it normal this on Ubuntu?

[rchiodo]

We don't log all of the directories that we index.

However, if you goto def on the BasePermission, it should show something like so:

class BasePermission(metaclass=BasePermissionMetaclass):
    """
    A base class from which all permission classes should inherit.
    """

BasePermissionClass should look like this:

class BasePermissionMetaclass(OperationHolderMixin, type):
    pass

And OperationHolderMixin should have the __or__ operator:

class OperationHolderMixin:
    def __and__(self, other):
        return OperandHolder(AND, self, other)

    def __or__(self, other):
        return OperandHolder(OR, self, other)

    def __rand__(self, other):
        return OperandHolder(AND, other, self)

    def __ror__(self, other):
        return OperandHolder(OR, other, self)

    def __invert__(self):
        return SingleOperandHolder(NOT, self)

If goto def isn't working or none of that is the same, that would explain the problem.

[sevdog]

@rchiodo actually the "go to definition" command only works only if I hover the package name (permissions), it does not work if I hover the BasePermission symbol without the permissions.py file opened.

When I open the source of site-packages/rest_framework/permissions.py the logs shows it get analyzed however the error remains on my console:

PS: there is no need to highlight the code of django rest framework.

[rchiodo]

What are these settings (in your user settings.json)?

  "python.analysis.indexing": true,
  "python.analysis.packageIndexDepths": [
    {
      "name": "django",
      "depth": 4,
      "includeAllSymbols": true
    }
  ],

If indexing is off, try turning it on. If you have any custom packageIndexDepths, remove them.

[sevdog]

@rchiodo adding that element to my settings does not change anything in my console.

My settings are those specified in the screenshot posted here.

Indexing has always been "on".

I tried setting "python.analysis.typeCheckingMode": "strict" and I got a nice hint of what is happening in this environment:

I finally found that for some reason the setting "python.analysis.useLibraryCodeForTypes" was turned off on user-profile (while it should be have on by default!), turning it on again solved the problem. That was the real issue behind this.

[rchiodo]

That makes sense. We don't import 3rd party library imports with that setting off.

For pyright this defaults to off, because type stubs typically have better type information, and pyright tries to be as deterministic as possible.

For pylance this defaults to true because when in the IDE, you probably want any information you can get while typing.

I wonder if there are stubs for django rest framework somewhere.

Oh yes there are: https://github.com/typeddjango/djangorestframework-stubs

Perhaps we can ship those to prevent this from happening to other people.

[rchiodo]

I created a discussion item to ship the django rest framework stubs: https://github.com/microsoft/pylance-release/discussions/3954