Implement testing such that it runs in VS Code and can be run in CI

[MBARIMike]

Following directions at https://djangostars.com/blog/django-pytest-testing/.

[MBARIMike]

Am able to execute pytest in a zsh terminal and the 21 tests pass.

Executing in the VS Code UI (the Erlenmeyer flask icon) I get all kinds of errors:

>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       django.db.utils.OperationalError: could not connect to server: Connection refused
E           Is the server running on host "localhost" (127.0.0.1) and accepting
E           TCP/IP connections on port 5432?
E       could not connect to server: Cannot assign requested address
E           Is the server running on host "localhost" (::1) and accepting
E           TCP/IP connections on port 5432?

/usr/local/lib/python3.8/dist-packages/psycopg2/__init__.py:127: OperationalError
=============================== warnings summary ===============================
smdb/tests.py::test_user_create
  /usr/local/lib/python3.8/dist-packages/django/db/backends/postgresql/base.py:304: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.

[MBARIMike]

With https://github.com/mbari-org/SeafloorMappingDB/pull/26 the problems above are fixed.

[MBARIMike]

Running pytest --cov in a zsh terminal now gives:

➜  /app pytest --cov
Test session starts (platform: linux, Python 3.8.5, pytest 6.2.3, pytest-sugar 0.9.4)
django: settings: config.settings.local (from ini)
rootdir: /app, configfile: pytest.ini
plugins: Faker-8.10.3, cov-2.12.1, django-4.2.0, sugar-0.9.4
collecting ... 
 smdb/tests.py ✓                                                                                                               5% ▌         
 smdb/users/tests/test_admin.py ✓✓✓✓                                                                                          24% ██▍       
 smdb/users/tests/test_drf_urls.py ✓✓✓                                                                                        38% ███▊      
 smdb/users/tests/test_drf_views.py ✓✓                                                                                        48% ████▊     
 smdb/users/tests/test_forms.py ✓                                                                                             52% █████▎    
 smdb/users/tests/test_models.py ✓                                                                                            57% █████▊    
 smdb/users/tests/test_urls.py ✓✓✓                                                                                            71% ███████▎  
 smdb/users/tests/test_views.py ✓✓✓✓✓✓                                                                                       100% ██████████
============================================================= warnings summary =============================================================
smdb/users/tests/test_admin.py::TestUserAdmin::test_changelist
smdb/users/tests/test_admin.py::TestUserAdmin::test_search
smdb/users/tests/test_admin.py::TestUserAdmin::test_add
smdb/users/tests/test_admin.py::TestUserAdmin::test_view_user
  /usr/local/lib/python3.8/dist-packages/whitenoise/base.py:115: UserWarning: No directory at: /app/staticfiles/
    warnings.warn(u"No directory at: {}".format(root))

smdb/users/tests/test_forms.py::TestUserCreationForm::test_username_validation_error_msg
  /app/smdb/users/tests/test_forms.py:39: RemovedInDjango40Warning: django.utils.translation.ugettext_lazy() is deprecated in favor of django.utils.translation.gettext_lazy().
    assert form.errors["username"][0] == _("This username has already been taken.")

-- Docs: https://docs.pytest.org/en/stable/warnings.html

----------- coverage: platform linux, python 3.8.5-final-0 -----------
Name                               Stmts   Miss  Cover
------------------------------------------------------
smdb/__init__.py                       2      0   100%
smdb/admin.py                         39      0   100%
smdb/conftest.py                       9      0   100%
smdb/contrib/__init__.py               0      0   100%
smdb/contrib/sites/__init__.py         0      0   100%
smdb/models.py                       131     25    81%
smdb/users/__init__.py                 0      0   100%
smdb/users/admin.py                   13      0   100%
smdb/users/api/serializers.py          8      0   100%
smdb/users/api/views.py               18      0   100%
smdb/users/apps.py                    10      0   100%
smdb/users/forms.py                   11      0   100%
smdb/users/models.py                  10      0   100%
smdb/users/urls.py                     4      0   100%
smdb/users/views.py                   26      0   100%
smdb/utils/__init__.py                 0      0   100%
smdb/utils/context_processors.py       3      0   100%
smdb/views.py                         25     14    44%
------------------------------------------------------
TOTAL                                309     39    87%

Results (4.40s):
      21 passed

We need to bring smdb/models.py and smdb/views.py up to 100%.

[MBARIMike]

Reading some posts and docs:

• https://medium.com/analytics-vidhya/how-to-unit-test-django-views-models-with-pytest-f44ea2c6bdc2
• https://pytest-django.readthedocs.io/en/latest/

I think I can achieve complete models.py test coverage by using the django rest framework to exercise CRUD operations on every model class. Though this tutorial uses Django's UnitTest (rather that pytest) it looks like a useful resource:

https://realpython.com/test-driven-development-of-a-django-restful-api/

I also need a test for the home "overview first" view that's been implemented.

Another doc: https://blog.logrocket.com/django-rest-framework-build-an-api-in-15-minutes/

[MBARIMike]

With https://github.com/mbari-org/SeafloorMappingDB/pull/26 tests are working now in VS Code:

[MBARIMike]

Adding one model class (Person) to the test requires modifying quite a few files:

    modified:   smdb/config/api_router.py
    modified:   smdb/smdb/api/serializers.py
    modified:   smdb/smdb/api/views.py
    modified:   smdb/smdb/conftest.py
    modified:   smdb/smdb/tests/factories.py
    modified:   smdb/smdb/tests/test_admin.py
    modified:   smdb/smdb/tests/test_drf_urls.py

I suppose this is the consequence of a deeply object oriented architecture.

[MBARIMike]

Test coverage is a little better:

Name                               Stmts   Miss  Cover
------------------------------------------------------
smdb/__init__.py                       2      0   100%
smdb/admin.py                         38      0   100%
smdb/api/serializers.py               14      0   100%
smdb/api/views.py                     19      0   100%
smdb/apps.py                           3      0   100%
smdb/conftest.py                      18      0   100%
smdb/contrib/__init__.py               0      0   100%
smdb/contrib/sites/__init__.py         0      0   100%
smdb/models.py                       131     18    86%
smdb/users/__init__.py                 0      0   100%
smdb/users/admin.py                   13      0   100%
smdb/users/api/serializers.py          8      0   100%
smdb/users/api/views.py               18      0   100%
smdb/users/apps.py                    10      0   100%
smdb/users/forms.py                   11      0   100%
smdb/users/models.py                  10      0   100%
smdb/users/urls.py                     4      0   100%
smdb/users/views.py                   26      0   100%
smdb/utils/__init__.py                 0      0   100%
smdb/utils/context_processors.py       3      0   100%
smdb/views.py                         24     14    42%
------------------------------------------------------
TOTAL                                352     32    91%

Results (9.93s):
      28 passed

Still not 100%, but framework is in place to add more tests and get CI working.

[MBARIMike]

Closing this Issue and staring new ones for follow-on work.

[MBARIMike]

Can't get testing to work in VS Code on my desktop. Getting these kind of errors:

python /root/.vscode-server/extensions/ms-python.python-2021.8.1105858891/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /app -s --cache-clear
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
django: settings: config.settings.test (from ini)
rootdir: /app, configfile: smdb/pytest.ini
plugins: django-4.2.0, cov-2.12.1, Faker-8.11.0, sugar-0.9.4
collected 28 items

smdb/tests/test_admin.py EEE                                             [ 10%]
smdb/tests/test_drf_urls.py EEEE                                         [ 25%]
smdb/users/tests/test_admin.py EEEE                                      [ 39%]
smdb/users/tests/test_drf_urls.py EEE                                    [ 50%]
smdb/users/tests/test_drf_views.py EE                                    [ 57%]
smdb/users/tests/test_forms.py E                                         [ 60%]
smdb/users/tests/test_models.py E                                        [ 64%]
smdb/users/tests/test_urls.py EEE                                        [ 75%]
smdb/users/tests/test_views.py EEEEEE                                    [ 96%]
smdb/tests/test_urls.py .                                                [100%]

==================================== ERRORS ====================================
___________ ERROR at setup of TestAdmin.test_missiontype_changelist ____________

…
self = <django.contrib.gis.db.backends.postgis.base.DatabaseWrapper object at 0x7fe84f029f10>
conn_params = {'database': 'postgres', 'host': 'localhost', 'password': '************', 'port': 5432, ...}
…
>       conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
E       psycopg2.OperationalError: could not connect to server: Connection refused
E           Is the server running on host "localhost" (127.0.0.1) and accepting
E           TCP/IP connections on port 5432?
E       could not connect to server: Cannot assign requested address
E           Is the server running on host "localhost" (::1) and accepting
E           TCP/IP connections on port 5432?
…
Error: Error: cannot open vscode-remote://dev-container%2B2f55736572732f6d6363616e6e2f4769744875622f536561666c6f6f724d617070696e674442/app/smdb/smdb/smdb/tests/
…

It's like somehow the proper settings aren't being used by VS Code on my desktop. Executing pytest from /app/smdb in VS Code's shell works fine on my desktop.

May be related to the Issue discussed here: https://github.com/microsoft/vscode-python/issues/8678

[MBARIMike]

Note that this problem cropped up after the container's /app context was changed from ./smdb to ./ as part of https://github.com/mbari-org/SeafloorMappingDB/issues/33, in particular this commit. Everything still works with pytest on the command line and in CI, it's just the VS Code execution that's broken.

[MBARIMike]

Now on my laptop when trying to run test in debug mode I get this error:

Error: Error: cannot open vscode-remote://dev-container%2B2f55736572732f6d6363616e6e2f4769744875622f536561666c6f6f724d617070696e674442/app/smdb/smdb/smdb/tests/test_drf_urls.py.

It seems that there are one too many smdbs in /app/smdb/smdb/smdb/tests/test_drf_urls.py.

[MBARIMike]

Tests in VS Code still failing:

Running test with arguments: --rootdir /app --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-257400157EBx0Y5.xml /app/smdb
Current working directory: /app/smdb
Workspace directory: /app
Run completed, parsing output
Test result not found for: ./smdb/tests/test_admin.py::TestAdmin::test_missiontype_changelist
...

Maybe this milestone will fix it: https://github.com/microsoft/vscode-python/issues/17242