Feature suggestion: run Django unittests

[DonJayamanne]

From @jvaesteves on January 16, 2017 22:28

Django unittest extends from unittests, but you can't run as the former on this extension. For this, you need to "manage.py test", but it would be awesome if there were those same shortcuts.

Copied from original issue: DonJayamanne/pythonVSCode#648

[djbrown]

on a new, minimal django proect (polls app from tutorial), the call for discovering tests looks like this for me: C:\Program Files\Python39\python.exe" ~\.vscode\extensions\ms-python.python-2022.18.2\pythonFiles\testing_tools\unittest_discovery.py . test*.py

so the vscode-python plugin calls its own disovery script: unittest_discovery.py

settings.json looks like this:

{
    "python.testing.unittestArgs": [
        "-v",
        "-p",
        "test*.py"
    ],
    "python.testing.pytestEnabled": false,
    "python.testing.unittestEnabled": true
}

full output (vscode > output > python)

``` > "C:\Program Files\Python39\python.exe" ~\.vscode\extensions\ms-python.python-2022.18.2\pythonFiles\testing_tools\unittest_discovery.py . test*.py сwd: . [ERROR 2023-1-9 18:29:1.465]: Error discovering unittest tests: Failed to import test module: polls.tests Traceback (most recent call last): File "C:\Program Files\Python39\lib\unittest\loader.py", line 436, in _find_test_path module = self._get_module_from_name(name) File "C:\Program Files\Python39\lib\unittest\loader.py", line 377, in _get_module_from_name __import__(name) File "c:\Users\myusername\projects\django-vscode\polls\tests.py", line 6, in from .models import Question File "c:\Users\myusername\projects\django-vscode\polls\models.py", line 4, in class Question(models.Model): File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\db\models\base.py", line 127, in __new__ app_config = apps.get_containing_app_config(module) File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\apps\registry.py", line 260, in get_containing_app_config self.check_apps_ready() File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\apps\registry.py", line 137, in check_apps_ready settings.INSTALLED_APPS File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\conf\__init__.py", line 92, in __getattr__ self._setup(name) File "C:\Users\myusername\AppData\Roaming\Python\Python39\site-packages\django\conf\__init__.py", line 72, in _setup raise ImproperlyConfigured( django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings. ```

similarly as @jvaesteves posted on January 19, 2017 12:13

ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

[brettcannon]

Looping in @eleanorjboyd and @karthiknadig to look into the questions that @carltongibson is asking.

[mh-firouzjah]

By using this fork of LittleFoxTeam.vscode-python-test-adapter extension the problem is fixed.

• there is a pull request

this is working by adding the following code to ~/.vscode/extensions/littlefoxteam.vscode-python-test-adapter-<version>/out/src/unittest/unittestTestRunner.js.

DON'T TRY AT HOME

from pathlib import Path
from ast import literal_eval
if Path(start_dir + "/manage.py").is_file():
    with open(start_dir + "/manage.py", "r") as management_file:
        contents = management_file.readlines()
        if any(True for line in contents if line.strip().replace('"""', '') == "Django\'s command-line utility for administrative tasks."):
            print("django management file found!")
            for line in contents:
                if line.strip().startswith("os.environ.setdefault"):
                    try:
                        literal_eval(line.strip().replace('os.environ.setdefault("DJANGO_SETTINGS_MODULE",', "", 1))
                    except:
                        pass
                    else:
                        eval(line.strip()) # this is the not recommended part!!!
                        try:
                            import django
                            django.setup()
                        except ModuleNotFoundError:
                            pass

[mh-firouzjah]

I have submitted a new pull request

• I'm not sure if it's perfect or not, but it works for my expectations

[carltongibson]

Replying to the recent comments:

import django
django.setup()

I didn't look at the details of the workarounds here but, just from a Django perspective, calling django.setup() is going to be needed. It requires an import path to a settings module, either via DJANGO_SETTINGS_MODULE env var or a --settings CLI flag to django-admin. (The default manage.py file sets a default for DJANGO_SETTINGS_MODULE to the settings file created with the startproject command, so that's not a bad fallback value.)

[mh-firouzjah]

@carltongibson I'm doing my best to make the changes more reliable and stable.(a new commit has been published)

[dimaulupov]

Hi, we have solved this problem in our project.
It needs a bit more than running django.setup().
Our code runs code below and full vscode testing pane works okay.

import django
from django.apps import apps
from django.test.utils import setup_databases, setup_test_environment
if not apps.ready:
    django.setup() # Normal Django setup
    setup_test_environment() # This does a lot of stuff inside Django tests
    # The next one is very important: it creates test databases and changes settings.DATABASES to point to them
    # otherwise tests will run against live database.
    setup_databases(verbosity=1, interactive=False, keepdb=True)
    # keepdb probably should be a setting inside vscode.
    # Our project takes an hour to run migrations from scratch, so we need keepdb, 
    # but normally no one wants to keep test databases around.

As a note if anyone is planning to do it before it is implemented in the extension.
This all is running in every <app>/tests/__init__.py.
Because if you are using something like Django Factory, which touches Django models on import, you'll get an exception just because discovery imported <app>/test/factories.py directly.
But if extension sets everything up - this won't be needed.
Also we run it only if there is "testlauncher.py" or "unittest_discovery" in sys.argv So this workaround does not kick in under normal ./manage.py test

[carltongibson]

This all is running in every <app>/tests/__init__.py.

In the extension, it should be sufficient to do once this before kicking off the test discovery… 🤔

[eleanorjboyd]

Hello! I have begun to review this question of what VS Code would need to get Django test discovery and run working. So far I had a few questions that came up during exploring documentation and your repo.

It looks like the DiscoverRunner is going to be most useful to us as we could create it then call its methods. What steps need to be taken before we can successfully call DiscoverRunner.run_tests() ? From my understanding , when comparing the functionality found in DiscoverRunner to that of the test method found in manage.py, It seems that the test method handles both interpreting command line args and then calling DiscoverRunner.run_tests() - is this correct? Are these command line args then applied to the DiscoverRunner object? If you could explain a little more how the args are processed that would be helpful.

Next I saw that the method DiscoverRunner.build_suite returns either unittest.TestSuite or ParallelTestSuite what is the difference? Could you also explain more the need for the **kwargs as an argument?

For running tests, I see the method DiscoverRunner.run_tests method which seems to only return the number of tests which failed. What is the return value of DiscoverRunner.run_suite, does it give which tests failed and an error message? If not where could I get that information after run?

Generally I want to understand the key differences between Django tests and Unittests and if the Django test objects can just be stored as test_ids/labels (these seems to be paths) and unittest.TestSuite. Also I want to understand how the environment setup and command line args work- is there an easy method that would allow us to set command line args like how it is done in the test method found in manage.py?

Thank you and I look forward to working with you all!

[carltongibson]

Hi @eleanorjboyd. Thanks for picking this up. It would be very exciting to get this working! 🤩

I want to understand how the environment setup and command line args work- is there an easy method that would allow us to set command line args like how it is done in the test method found in manage.py?

Naïvely, the basics are setting the DJANGO_SETTINGS_MODULE environment variable, and then calling django.setup(). As noted in comments above here, there are a couple of other steps for the tests, like setting up the test database.

If you look at the ManagementUtility class (and the execute_from_command_line helper) — I think you'd have most of what you're asking for there. (See also though the call_command function (docs).)

The ManagementUtility.execute() calls django.setup() and dispatches to the test command. (So, going this route if you set the DJANGO_SETTINGS_MODULE environment variable, the rest is handled for you.)

However you need the results... 🤔

For running tests, I see the method DiscoverRunner.run_tests method which seems to only return the number of tests which failed. What is the return value of DiscoverRunner.run_suite, does it give which tests failed and an error message? If not where could I get that information after run?

So DiscoverRunner.run_suite gives you the TextTestResult that I think you're after? You're right we discard that currently in run_tests, so for the first attempt I'd likely override run_suite to keep a reference to the result, and then hopefully you could use run_tests() from your subclass, as the test command does.

• This runs the database setup and such.
• We'd certainly look at keeping that result reference for you in Django's development branch if that's what you need to make this work long-term!

Next I saw that the method DiscoverRunner.build_suite returns either unittest.TestSuite or ParallelTestSuite what is the difference? Could you also explain more the need for the **kwargs as an argument?

ParallelTestSuite hides the fact that we're running multiple processes from the test runner, suppressing subprocess output, and combining into a single result. The build_suite() kwargs have never been used, as far as I can tell. (It's looks to me like a case of YAGNI but I'd have to dig a lot to say more... — they should be safe to ignore.)

Generally I want to understand the key differences between Django tests and Unittests and if the Django test objects can just be stored as test_ids/labels (these seems to be paths) and unittest.TestSuite.

At a high-level, I'd say we (cough) just wrap unittest, and hopefully there's all the API you need there (somewhere) — As I say, we're happy to look at adding the hooks you need here.

Generally labels are import paths to tests: myapp.tests.MyTestCase.test_my_method, but could be a directory path too. (build_suite reference).

Also I want to understand how the environment setup and command line args work- is there an easy method that would allow us to set command line args like how it is done in the test method found in manage.py?

Hopefully the ManagementUtility stuff at the top there is what you need? For running the Django test suite itself we configure INSTALLED_APPS and other settings by hand (because they can vary depending on what tests you're running) — but that is equivalent to running django.setup().

Schematically, I would think something like this would be Pretty Close™:

os.environ["DJANGO_SETTINGS_MODULE"] = "user_project.settings"
django.setup()
runner = DiscoverRunner(**required_kwargs)  # 😬 Need to check
runner.run_tests(["user_app.tests"])

Does that give you the guidance you need? Let me know!

I don't know my way around the extension, but happy to look at a branch if you can give me the dummy's guide steps to get it running.

Thank you and I look forward to working with you all!

You too! Thanks again. 💃

[eleanorjboyd]

Hello @carltongibson, great to meet you! Thank you for your in-depth reply! I am currently thinking on the next steps and talking with the team so I will get back to you soon. Your answers are extremely helpful in narrowing down what is needed and how to create compatibility. I am excited about the shared types with unittest and how this can make the integration more seamless.

Thanks!

[antoniogamizbadger]

So.. any update on this? I tried setting up our company project with VSCode and the extension is still unable to find django test modules.

[khamaileon]

Apparently the next step is here and we need votes to get things going. If the 300+ people who liked this post could make their voices heard.

[carltongibson]

@khamaileon Nice. Good spot. I didn't even know about that other issue.

[mh-firouzjah]

Apparently the next step is here and we need votes to get things going. If the 300+ people who liked this post could make their voices heard.

As long as the proposed procedure is taking approval, I have tested a new idea, and it worked well for me. I am awaiting feedback and information on whether it works properly for you and others, or if there are any issues.

I expected that after running the tests, there would be a change in the main project database Which it did not and the main database is clear; because in this approach, I did not use a CustomTestRunner.

NOTE: This method requires fewer changes in the vscode extension and is also easier to modify the django_settings_module for each project, especially when we have multiple modules as settings for different scenarios such as development, production, and so on.

1- Vscode Python Extension Requirement edit the __init__.py module in ~/.vscode/extensions/ms-python.python-VERSION/pythonFiles/unittestadapter/__init__.py and add the code below:

try:
    import django
    django.setup()
except:
    pass

2- Django Project Requirement create a .env file and inside of it declare DJANGO_SETTINGS_MODULE. an example is:

DJANGO_SETTINGS_MODULE=my_django_project_name.settings

[khamaileon]

@mh-firouzjah I tried but it didn't work out.

Here's my vscode settings:

{
  "editor.codeActionsOnSave": {
    "source.organizeImports": "explicit"
  },
  "editor.formatOnSave": true,
  "python.defaultInterpreterPath": "~/.virtualenvs/acme-api/bin/python",
  "python.envFile": "${workspaceFolder}/.env.local",
  "editor.rulers": [100],
  "python.testing.unittestArgs": ["-v", "-s", "./src", "-p", "test*.py"],
  "python.testing.pytestEnabled": false,
  "python.testing.unittestEnabled": true
}

And here's the kind of output I got:

2024-02-06 10:36:32.276 [info] Telemetry level is off
2024-02-06 10:36:32.276 [info] Experiments are disabled, only manually opted experiments are active.
2024-02-06 10:36:32.276 [info] Default formatter is set to ms-python.black-formatter for workspace /home/me/workspace/acme/mars-api
2024-02-06 10:36:32.276 [info] Test server listening.
2024-02-06 10:36:32.276 [info] VS Code was launched from an activated environment: 'mars-api', selecting it as the interpreter for workspace.
2024-02-06 10:36:32.276 [info] Python interpreter path: ~/.virtualenvs/mars-api/bin/python
2024-02-06 10:36:34.315 [info] Starting Pylance language server.
2024-02-06 10:36:43.381 [info] Discover tests for workspace name: mars-api - uri: /home/me/workspace/acme/mars-api
2024-02-06 10:36:43.405 [info] > . ~/.virtualenvs/mars-api/bin/activate && echo '<some_uuid>' && python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/printEnvVariables.py
2024-02-06 10:36:43.405 [info] shell: bash
2024-02-06 10:36:43.457 [info] > ~/.virtualenvs/mars-api/bin/python ~/.vscode/extensions/ms-python.python-2024.0.0/pythonFiles/testing_tools/unittest_discovery.py ./src test*.py
2024-02-06 10:36:43.457 [info] cwd: .
2024-02-06 10:36:44.068 [error] Error discovering unittest tests:
 Failed to import test module: account.tests
Traceback (most recent call last):
  File "/usr/lib/python3.12/unittest/loader.py", line 394, in _find_test_path
    module = self._get_module_from_name(name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/unittest/loader.py", line 337, in _get_module_from_name
    __import__(name)
  File "/home/me/workspace/acme/mars-api/src/account/tests.py", line 15, in <module>
    from account.models import Session, User
  File "/home/me/workspace/acme/mars-api/src/account/models.py", line 3, in <module>
    from django.contrib.auth.models import AbstractUser
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/contrib/auth/base_user.py", line 58, in <module>
    class AbstractBaseUser(models.Model):
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/db/models/base.py", line 129, in __new__
    app_config = apps.get_containing_app_config(module)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/apps/registry.py", line 260, in get_containing_app_config
    self.check_apps_ready()
  File "/home/me/.virtualenvs/mars-api/lib/python3.12/site-packages/django/apps/registry.py", line 138, in check_apps_ready
    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.

DJANGO_SETTINGS_MODULE=project.settings is read correctly, otherwise I'm told it's necessary.

The problem may be that my code is in a src folder and not in the root. 🤷‍♂️

[mh-firouzjah]

hi @khamaileon, yes, DJANGO_SETTINGS_MODULE is read. but for the error you got it seems that django.setup() had not been called at the beginning of the process. being in a subdirectory could be the case but if you also have error when using manage.py check/runserver/... otherwise I don't think thats the problem here.

[khamaileon]

Yet I'm able to launch it with the manage command inside vscode and outside (from the src folder). Here is my launch.json config file.

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: Django",
      "type": "python",
      "request": "launch",
      "program": "${workspaceFolder}/src/manage.py",
      "args": ["runserver"],
      "django": true,
      "envFile": "${workspaceFolder}/.env.local",
      "justMyCode": true
    }
  ]
}

[mardukbp]

Thank you @mh-firouzjah! Your solution works great :)