Follow zero internal warning in unit tests

Body

This is follow up task after https://github.com/apache/airflow/pull/38504, we have quite a few tests which raise one of the internal deprecation warning:

airflow.exceptions.AirflowProviderDeprecationWarning
airflow.exceptions.RemovedInAirflow3Warning
airflow.utils.context.AirflowContextDeprecationWarning

All current exclusions which should be resolved stored into the tests/deprecations_ignore.yml in format test node id: {relative path to the test}::{ClassName}::{test_case} or {relative path to the test}::{test_case} without parametrised part

How to resolve specific test suite

Remove /commented out specific line in exclusion list a have a look on error by run removed node id,

root@e68ed83041ce:/opt/airflow# pytest tests/decorators/test_python_virtualenv.py::TestPythonVirtualenvDecorator::test_python_3
======================================================================= test session starts ========================================================================
platform linux -- Python 3.8.19, pytest-7.4.4, pluggy-1.4.0 -- /usr/local/bin/python
cachedir: .pytest_cache
rootdir: /opt/airflow
configfile: pyproject.toml
plugins: custom-exit-code-0.3.0, instafail-0.5.0, rerunfailures-14.0, asyncio-0.23.6, cov-5.0.0, xdist-3.5.0, timeouts-1.2.1, anyio-4.3.0, icdiff-0.9, mock-3.14.0, requests-mock-1.11.0, time-machine-2.14.1
asyncio: mode=strict
setup timeout: 0.0s, execution timeout: 0.0s, teardown timeout: 0.0s
collected 1 item                                                                                                                                                   

tests/decorators/test_python_virtualenv.py::TestPythonVirtualenvDecorator::test_python_3 FAILED                                                              [100%]

============================================================================= FAILURES =============================================================================
___________________________________________________________ TestPythonVirtualenvDecorator.test_python_3 ____________________________________________________________

self = <tests.decorators.test_python_virtualenv.TestPythonVirtualenvDecorator object at 0xffff855dd760>
dag_maker = <tests.conftest.dag_maker.<locals>.DagFactory object at 0xffff8593da30>

    def test_python_3(self, dag_maker):
        @task.virtualenv(python_version=3, use_dill=False, requirements=["dill"])
        def f():
            import sys

            print(sys.version)
            try:
                {}.iteritems()
            except AttributeError:
                return
            raise Exception

        with dag_maker():
>           ret = f()

tests/decorators/test_python_virtualenv.py:171: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
airflow/decorators/base.py:372: in __call__
    op = self.operator_class(
airflow/models/baseoperator.py:483: in apply_defaults
    result = func(self, **kwargs, default_args=default_args)
airflow/models/baseoperator.py:483: in apply_defaults
    result = func(self, **kwargs, default_args=default_args)
airflow/decorators/python.py:52: in __init__
    super().__init__(
airflow/models/baseoperator.py:483: in apply_defaults
    result = func(self, **kwargs, default_args=default_args)
airflow/decorators/base.py:257: in __init__
    super().__init__(task_id=task_id, **kwargs_to_upstream, **kwargs)
airflow/models/baseoperator.py:483: in apply_defaults
    result = func(self, **kwargs, default_args=default_args)
airflow/operators/python.py:620: in __init__
    warnings.warn(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <airflow.utils.decorators._autostacklevel_warn object at 0xffff85ebb820>
message = 'Passing non-string types (e.g. int or float) as python_version is deprecated. Please use string value instead.'
category = <class 'airflow.exceptions.RemovedInAirflow3Warning'>, stacklevel = 2, source = None

    def warn(self, message, category=None, stacklevel=1, source=None):
>       self.warnings.warn(message, category, stacklevel + 2, source)
E       airflow.exceptions.RemovedInAirflow3Warning: Passing non-string types (e.g. int or float) as python_version is deprecated. Please use string value instead.

airflow/utils/decorators.py:108: RemovedInAirflow3Warning
---------------------------------------------------------------------- Captured stdout setup -----------------------------------------------------------------------
========================= AIRFLOW ==========================
Home of the user: /root
Airflow home /root/airflow
Skipping initializing of the DB as it was initialized already.
You can re-initialize the database by adding --with-db-init flag when running tests.
[2024-03-31T10:06:44.945+0000] {dagbag.py:540} INFO - Filling up the DagBag from /dev/null
------------------------------------------------------------------------ Captured log setup ------------------------------------------------------------------------
INFO     airflow.models.dagbag.DagBag:dagbag.py:540 Filling up the DagBag from /dev/null
===================================================================== short test summary info ======================================================================
FAILED tests/decorators/test_python_virtualenv.py::TestPythonVirtualenvDecorator::test_python_3 - airflow.exceptions.RemovedInAirflow3Warning: Passing non-string types (e.g. int or float) as python_version is deprecated. Please use string value instead.

After that need to determine which kind of a problem here

Option 1: It might be test deprecation functional without explicit usage of pytest.warn. This case is pretty simple and required to add pytest.warn context manager wit appropriate matcher.

Option 2: Use deprecated functional in tests, time moves on and not all test updated. In this case need to rewrite test to avoid use deprecated part

Option 3: Use deprecated functional in tests utilities and fixtures, the same as option 2, however it might spread across multiple tests suits, replace/add functional to use not deprecated functional it this utilities/fixtures. This might required to use full test needed label in pr, If you not a Commiter or PMC please ask to add this label.

Option 4: Incorrect depreciation in code base. This one could be cause if one part of functional deprecated in code base, but the other still use it. There is no a universal solution for this situation, it might required to avoid use deprecated functional in codebase or even undeprecate.

Other cases: It might be other cases, feel free to share you experience with non listed above in this issue or into the Slack

[!TIP] Feel free to ask a suggestion/helps in slack channels #contributors or #new-contributors

[!NOTE] There main target of this task it resolve deprecation warnings, but not hide all of them under pytest.warn, use this context manager, as well as warnings.catch_warnings context manager only for test suites around deprecated functional

[!WARNING] Please do not mixin changes in code base and only tests into the same PR. Better to split it into the separate PRs.

Committer

[X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

apache / airflow