search

comic / grand-challenge.org

A platform for end-to-end development of machine learning solutions in biomedical imaging
https://grand-challenge.org
Apache License 2.0
176 stars 51 forks source link

Fix test warnings #2939

Closed jmsmkn closed 1 year ago

jmsmkn commented 1 year ago

Lots of warnings from a full test run now, these need to be handled.

=============================== warnings summary ===============================
../opt/poetry/.venv/lib/python3.10/site-packages/simple_history/__init__.py:1
../opt/poetry/.venv/lib/python3.10/site-packages/simple_history/__init__.py:1
../opt/poetry/.venv/lib/python3.10/site-packages/simple_history/__init__.py:1
  /opt/poetry/.venv/lib/python3.10/site-packages/simple_history/__init__.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pkg_resources import DistributionNotFound, get_distribution

tests/conftest.py:79
tests/conftest.py:79
  /app/tests/conftest.py:79: UserWarning: Setting DJANGO_ALLOW_ASYNC_UNSAFE for playwright support
    warnings.warn(  # noqa: B028

tests/evaluation_tests/test_models.py: 251 warnings
tests/flatpages_tests/test_views.py: 2 warnings
tests/publications_tests/test_views.py: 3 warnings
tests/verification_tests/test_forms.py: 12 warnings
tests/profiles_tests/test_models.py: 5 warnings
tests/evaluation_tests/test_config.py: 14 warnings
tests/workstations_tests/test_signals.py: 4 warnings
tests/cases_tests/test_permissions.py: 26 warnings
tests/reader_studies_tests/test_permissions.py: 93 warnings
tests/hanging_protocols_tests/test_serializers.py: 8 warnings
tests/participants_tests/test_workflow.py: 30 warnings
tests/components_tests/test_tasks.py: 7 warnings
tests/cases_tests/test_background_tasks.py: 11 warnings
tests/workstations_tests/test_permissions.py: 171 warnings
tests/reader_studies_tests/test_api.py: 190 warnings
tests/workstations_tests/test_utils.py: 4 warnings
tests/evaluation_tests/test_permissions.py: 45 warnings
tests/admins_tests/test_permissions.py: 26 warnings
tests/algorithms_tests/test_views.py: 86 warnings
tests/github_tests/test_tasks.py: 5 warnings
tests/github_tests/test_views.py: 2 warnings
tests/profiles_tests/test_tasks.py: 2 warnings
tests/api_tokens_tests/test_views.py: 5 warnings
tests/components_tests/test_serializers.py: 12 warnings
tests/reader_studies_tests/test_models.py: 36 warnings
tests/cases_tests/test_serializers.py: 2 warnings
tests/participants_tests/test_links.py: 5 warnings
tests/components_tests/test_models.py: 27 warnings
tests/evaluation_tests/test_views.py: 89 warnings
tests/notifications_tests/test_utils.py: 6 warnings
tests/archives_tests/test_signals.py: 28 warnings
tests/groups_tests/test_views.py: 36 warnings
tests/reader_studies_tests/test_signals.py: 4 warnings
tests/workstations_tests/test_models.py: 25 warnings
tests/cases_tests/test_widget.py: 3 warnings
tests/cases_tests/test_api.py: 21 warnings
tests/participants_tests/test_participants.py: 28 warnings
tests/core_tests/test_tasks.py: 8 warnings
tests/workstations_tests/test_views.py: 53 warnings
tests/serving_tests/test_views.py: 29 warnings
tests/hanging_protocols_tests/test_forms.py: 4 warnings
tests/teams_tests/test_views.py: 122 warnings
tests/evaluation_tests/test_utils.py: 49 warnings
tests/common_tests/test_link_visibility.py: 91 warnings
tests/cases_tests/test_views.py: 25 warnings
tests/core_tests/test_signals.py: 25 warnings
tests/hanging_protocols_tests/test_permissions.py: 2 warnings
tests/pages_tests/test_pages.py: 191 warnings
tests/participants_tests/test_filters.py: 28 warnings
tests/workspaces_tests/test_views.py: 6 warnings
tests/core_tests/test_guardian.py: 40 warnings
tests/challenges_tests/test_forms.py: 4 warnings
tests/blogs_tests/test_views.py: 2 warnings
tests/challenges_tests/test_management_commands.py: 1 warning
tests/algorithms_tests/test_permissions.py: 43 warnings
tests/notifications_tests/test_views.py: 21 warnings
tests/cases_tests/test_forms.py: 7 warnings
tests/algorithms_tests/test_tasks.py: 25 warnings
tests/core_tests/test_permission_requests.py: 44 warnings
tests/evaluation_tests/test_results.py: 7 warnings
tests/admins_tests/test_filters.py: 13 warnings
tests/algorithms_tests/test_serializers.py: 22 warnings
tests/uploads_tests/test_views.py: 16 warnings
tests/evaluation_tests/test_admin.py: 2 warnings
tests/algorithms_tests/test_forms.py: 45 warnings
tests/evaluation_tests/test_tasks.py: 29 warnings
tests/uploads_tests/test_tasks.py: 5 warnings
tests/challenges_tests/test_views.py: 63 warnings
tests/algorithms_tests/test_filters.py: 15 warnings
tests/profiles_tests/test_2fa.py: 7 warnings
tests/evaluation_tests/test_forms.py: 41 warnings
tests/components_tests/test_views.py: 6 warnings
tests/forums_tests/test_signals.py: 2 warnings
tests/core_tests/integration_tests.py: 6 warnings
tests/archives_tests/test_serializers.py: 2 warnings
tests/challenges_tests/test_models.py: 53 warnings
tests/archives_tests/test_forms.py: 11 warnings
tests/challenges_tests/test_permissions.py: 21 warnings
tests/blogs_tests/test_signals.py: 12 warnings
tests/reader_studies_tests/test_filters.py: 2 warnings
tests/components_tests/test_forms.py: 2 warnings
tests/annotations_tests/test_models.py: 22 warnings
tests/emails_tests/test_views.py: 4 warnings
tests/organizations_tests/test_views.py: 7 warnings
tests/admins_tests/test_forms.py: 27 warnings
tests/products_tests/test_permissions.py: 2 warnings
tests/blogs_tests/test_models.py: 1 warning
tests/subdomain_tests/test_middleware.py: 10 warnings
tests/reader_studies_tests/test_forms.py: 102 warnings
tests/notifications_tests/test_tasks.py: 5 warnings
tests/notifications_tests/test_signals.py: 24 warnings
tests/uploads_tests/test_models.py: 7 warnings
tests/algorithms_tests/test_signals.py: 54 warnings
tests/profiles_tests/test_views.py: 8 warnings
tests/annotations_tests/test_serializers.py: 45 warnings
tests/api_tests/test_permissions.py: 1 warning
tests/archives_tests/test_permissions.py: 4 warnings
tests/core_tests/test_groups.py: 1 warning
tests/participants_tests/test_permissions.py: 45 warnings
tests/retina_api_tests/test_viewsets.py: 1605 warnings
tests/organizations_tests/test_signals.py: 8 warnings
tests/core_tests/test_fixtures.py: 23 warnings
tests/emails_tests/test_tasks.py: 22 warnings
tests/archives_tests/test_views.py: 27 warnings
tests/workstations_tests/test_api.py: 23 warnings
tests/reader_studies_tests/test_views.py: 27 warnings
tests/challenges_tests/test_tasks.py: 34 warnings
tests/algorithms_tests/test_models.py: 65 warnings
tests/challenges_tests/test_filters.py: 13 warnings
tests/documentation_tests/test_views.py: 9 warnings
tests/reader_studies_tests/test_templatetags.py: 3 warnings
tests/algorithms_tests/test_api.py: 6 warnings
tests/blogs_tests/test_permissions.py: 2 warnings
tests/workstation_config_tests/test_views.py: 5 warnings
tests/statistics_tests/test_views.py: 3 warnings
tests/hanging_protocols_tests/test_models.py: 26 warnings
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: UserFactory._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the UserFactory.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_permissions.py: 13 warnings
tests/hanging_protocols_tests/test_serializers.py: 1 warning
tests/algorithms_tests/test_views.py: 54 warnings
tests/components_tests/test_models.py: 5 warnings
tests/cases_tests/test_api.py: 1 warning
tests/core_tests/test_tasks.py: 1 warning
tests/serving_tests/test_views.py: 1 warning
tests/algorithms_tests/test_permissions.py: 3 warnings
tests/pages_tests/test_pages.py: 2 warnings
tests/algorithms_tests/test_tasks.py: 2 warnings
tests/algorithms_tests/test_serializers.py: 1 warning
tests/algorithms_tests/test_forms.py: 1 warning
tests/evaluation_tests/test_tasks.py: 2 warnings
tests/algorithms_tests/test_filters.py: 5 warnings
tests/evaluation_tests/test_forms.py: 1 warning
tests/algorithms_tests/test_signals.py: 15 warnings
tests/algorithms_tests/test_models.py: 40 warnings
tests/algorithms_tests/test_api.py: 2 warnings
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: AlgorithmJobFactory._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the AlgorithmJobFactory.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_permissions.py: 8 warnings
tests/hanging_protocols_tests/test_serializers.py: 1 warning
tests/evaluation_tests/test_views.py: 3 warnings
tests/archives_tests/test_signals.py: 16 warnings
tests/groups_tests/test_views.py: 10 warnings
tests/cases_tests/test_api.py: 6 warnings
tests/archives_tests/test_tasks.py: 1 warning
tests/serving_tests/test_views.py: 1 warning
tests/core_tests/test_signals.py: 4 warnings
tests/algorithms_tests/test_permissions.py: 2 warnings
tests/core_tests/test_permission_requests.py: 4 warnings
tests/evaluation_tests/test_tasks.py: 3 warnings
tests/archives_tests/test_forms.py: 4 warnings
tests/organizations_tests/test_views.py: 1 warning
tests/archives_tests/test_models.py: 1 warning
tests/archives_tests/test_permissions.py: 7 warnings
tests/organizations_tests/test_signals.py: 8 warnings
tests/archives_tests/test_views.py: 23 warnings
tests/evaluation_tests/test_models.py: 2 warnings
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ArchiveFactory._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ArchiveFactory.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_models.py::TestGetSitkImage::test_4d_mhd_object
tests/components_tests/test_management_commands.py::test_add_overlay_segments
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFile4D._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFile4D.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_models.py: 7 warnings
tests/cases_tests/test_serializers.py: 2 warnings
tests/cases_tests/test_views.py: 10 warnings
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFile._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFile.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/components_tests/test_models.py::test_default_validation[CHART-value30-expectation30-True]
tests/components_tests/test_models.py::test_default_validation[CHART-value29-expectation29-True]
tests/components_tests/test_models.py::test_default_validation[CHART-value30-expectation30-False]
tests/components_tests/test_models.py::test_default_validation[CHART-value29-expectation29-False]
  /opt/poetry/.venv/lib/python3.10/site-packages/jsonschema/_validators.py:341: DeprecationWarning: Automatically retrieving remote references can be a security vulnerability and is discouraged by the JSON Schema specifications. Relying on this behavior is deprecated and will shortly become an error. If you are sure you want to remotely retrieve your reference and that it is safe to do so, you can find instructions for doing so via referencing.Registry in the referencing documentation (https://referencing.readthedocs.org).
    errs = list(validator.descend(instance, subschema, schema_path=index))

tests/components_tests/test_models.py::test_validate_voxel_values
tests/components_tests/test_models.py::test_validate_voxel_values
tests/components_tests/test_models.py::test_validate_voxel_values
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFileTiff._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFileTiff.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile3D-kwargs4-cs3d]
tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile3D-kwargs4-cs]
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFile3D._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFile3D.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile2DGray16Bit-kwargs0-cs3d]
tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile2DGray16Bit-kwargs0-cs]
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFile2DGray16Bit._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFile2DGray16Bit.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile16Bit-kwargs3-cs3d]
tests/cases_tests/test_views.py::TestCSImageDetail::test_allowed[ImageFactoryWithImageFile16Bit-kwargs3-cs]
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ImageFactoryWithImageFile16Bit._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ImageFactoryWithImageFile16Bit.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/blogs_tests/test_views.py::test_author_add_button_visibility
tests/blogs_tests/test_signals.py::test_post_authors_permissions_signal[False]
tests/blogs_tests/test_signals.py::test_post_authors_permissions_signal[False]
tests/blogs_tests/test_signals.py::test_post_authors_permissions_signal[True]
tests/blogs_tests/test_signals.py::test_post_authors_permissions_signal[True]
tests/products_tests/test_permissions.py::TestObjectPermissionRequiredViews::test_permission_required_views
tests/blogs_tests/test_models.py::test_created_updated_when_published
tests/blogs_tests/test_permissions.py::TestObjectPermissionRequiredViews::test_permission_required_views
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: PostFactory._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the PostFactory.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/notifications_tests/test_views.py: 1 warning
tests/forums_tests/test_signals.py: 2 warnings
tests/challenges_tests/test_models.py: 2 warnings
tests/notifications_tests/test_signals.py: 11 warnings
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: TopicFactory._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the TopicFactory.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

tests/archives_tests/test_serializers.py::TestSerializers::test_serializer_valid[serializer_data0]
tests/archives_tests/test_serializers.py::TestSerializers::test_serializer_fields[serializer_data0]
  /opt/poetry/.venv/lib/python3.10/site-packages/factory/django.py:181: DeprecationWarning: ArchiveWithHangingProtocol._after_postgeneration will stop saving the instance after postgeneration hooks in the next major release.
  If the save call is extraneous, set skip_postgeneration_save=True in the ArchiveWithHangingProtocol.Meta.
  To keep saving the instance, move the save call to your postgeneration hooks or override _after_postgeneration.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
chrisvanrun commented 1 year ago

DeprecationWarning: pkg_resources is from package django-simple-history (v3.3.0). The warning hould be fixed in the next release of django-simple-history.

Relevant commit (2023-07-04): https://github.com/jazzband/django-simple-history/commit/5439c006b228fb86f3e0d5f4c0840f1277c8be3a

chrisvanrun commented 1 year ago

Rabbit hole for the validator warning:

/opt/poetry/.venv/lib/python3.10/site-packages/jsonschema/_validators.py:341: DeprecationWarning: Automatically retrieving remote references can be a security vulnerability and is discouraged by the JSON Schema specifications. Relying on this behavior is deprecated and will shortly become an error. If you are sure you want to remotely retrieve your reference and that it is safe to do so, you can find instructions for doing so via referencing. Registry in the referencing documentation

It boils down to adding this to the general schema for "CHART" type of Component-Interface Values: https://github.com/comic/grand-challenge.org/blob/7f987304983a7e718981d8d94f014d93b7638952/app/grandchallenge/components/schemas.py#L533-L536

Making the linked schema a static item isn't ideal since it is 31.000+ lines long (!).

As such perhaps the best approach is implementing the registry mentioned in the deprecation warning. However, the default jsonschema.validate does not seem to allow one to specify which registry to use... so we need to function one level lower and instantiate the ...Validator() objects ourselves... oh joy! More info here: https://python-jsonschema.readthedocs.io/en/stable/referencing/

Rabbit hole!

chrisvanrun commented 1 year ago

Actually, it seems like a passthrough occurs for *args, **kwargs for validate(): https://github.com/python-jsonschema/jsonschema/blob/2f5ff0b14c9ec317a64aef02898f7c44e81ec607/jsonschema/validators.py#L1304C45-L1304C45

chrisvanrun commented 1 year ago

I've created two PRs that should address ~99% of all the warnings.

jmsmkn commented 1 year ago

The remaining warning from django simple history/pkg_resources can be ignored in pytest.ini.