open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
5.6k stars 1.05k forks source link

Wrong logic for version check in API #18349

Open imishchuk-tsgs opened 1 month ago

imishchuk-tsgs commented 1 month ago

Affected module API

Describe the bug Current version check in openmetadata-airflow-apis/opendata_managed_apis/api/utils.py is done lexiographically, instead of proper version comparison

 if airflow_version >= "2.6":

This works fine for Airflow version 2.9.* but breaks for 2.10.*

Process ScanDagsTask-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/home/airflow/.local/lib/python3.12/site-packages/openmetadata_managed_apis/api/utils.py", line 116, in run
    scheduler_job = self._run_old_scheduler_job()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/openmetadata_managed_apis/api/utils.py", line 148, in _run_old_scheduler_job
    from airflow.jobs.scheduler_job import SchedulerJob
ModuleNotFoundError: No module named 'airflow.jobs.scheduler_job'

Airflow version

airflow version
2.10.2

To Reproduce

Expected behavior Version comparison is done correctly, i.e. 2.10.* is newer than 2.9.*

Version:

Additional context

c = "2.9"

if c >= "2.6":
    print("OK")
else:
    print("NOK")

OK
c = "2.10"

if c >= "2.6":
    print("OK")
else:
    print("NOK")

NOK

Fixed

from packaging import version

c = "2.10"

if version.parse(c) >= version.parse("2.6"):
    print("OK")
else:
    print("NOK")

OK
imishchuk-tsgs commented 1 month ago

https://github.com/open-metadata/OpenMetadata/pull/18352