search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.02k stars 14.27k forks source link

Status of testing of Apache Airflow 2.10.3rc2 #43441

Closed utkarsharma2 closed 2 days ago

utkarsharma2 commented 1 week ago

Body

We are kindly requesting that contributors to Apache Airflow RC 2.10.3rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

Thanks to all who contributed to the release (probably not a complete list!): @uranusjr @howardyoo @majorosdonat @ashb @rawwar @jonhspyro @kaxil @potiuk @pierrejeambrun @Lee-W @bbovenzi @ephraimbuddy @AutomationDev85 @vincbeck @jscheffl @k-slash @jason810496 @shahar1 @jedcunningham @romsharon98 @dabla @jmaicher @gopidesupavan @utkarsharma2 @TakawaAkirayo @josephangbc @brightview4578 @Joffreybvn @abhishekbhakat

Committer

raphaelauv commented 1 week ago

hi , the docker image are not yet available -> https://hub.docker.com/r/apache/airflow/tags?name=2.10.3

utkarsharma2 commented 1 week ago

hi , the docker image are not yet available -> https://hub.docker.com/r/apache/airflow/tags?name=2.10.3

@raphaelauv Thanks, I'm looking into it.

raphaelauv commented 1 week ago

I can't open from UI the variables,pools and connection ( no problem with 2.10.2 )

image

it fail with 

[2024-10-29T10:00:34.055+0000] {app.py:1744} ERROR - Exception on /variable/list/ [GET]
 Traceback (most recent call last):
   File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 2529, in wsgi_app
     response = self.full_dispatch_request()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1825, in full_dispatch_request
     rv = self.handle_user_exception(e)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1823, in full_dispatch_request
     rv = self.dispatch_request()
          ^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask/app.py", line 1799, in dispatch_request
     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/security/decorators.py", line 137, in wraps
     return f(self, *args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/views.py", line 550, in list
     widgets = self._list()
               ^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/baseviews.py", line 1186, in _list
     form = self.search_form.refresh()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask_appbuilder/forms.py", line 327, in refresh
     form = self(obj=obj)
            ^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/form.py", line 209, in __call__
     return type.__call__(cls, *args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/flask_wtf/form.py", line 73, in __init__
     super().__init__(formdata=formdata, **kwargs)
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/form.py", line 281, in __init__
     super().__init__(self._unbound_fields, meta=meta_obj, prefix=prefix)
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/form.py", line 49, in __init__
     field = meta.bind_field(self, unbound_field, options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/meta.py", line 28, in bind_field
     return unbound_field.bind(form=form, **options)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/fields/core.py", line 387, in bind
     return self.field_class(*self.args, **kw)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/home/airflow/.local/lib/python3.12/site-packages/wtforms/fields/core.py", line 133, in __init__
     for k, v in flags.items():
                 ^^^^^^^^^^^
 AttributeError: 'tuple' object has no attribute 'items'
pierrejeambrun commented 1 week ago

Thanks for reporting @raphaelauv this happens because we need to release the FAB provider to update the constraints of the RC to the latest FlaskAppBuilder. (latest release of WTForm broke Flask app builder).

potiuk commented 1 week ago

Thanks for reporting @raphaelauv this happens because we need to release the FAB provider to update the constraints of the RC to the latest FlaskAppBuilder. (latest release of WTForm broke Flask app builder).

Yeah. We should regenerate constraints and rebuild images as a final check after we release FAB provider.

thirtyseven commented 1 week ago

I have tested 2.10.3rc1 and confirmed that #42351 works.

k-slash commented 1 week ago

The same here for #42796 . It works under 2.10.3rc1, thanks !

jscheffl commented 1 week ago

From the VOTE of provider tests of @eladkal in https://lists.apache.org/thread/76jbfwzlh7vshl9rzmpk9okl5ot3bxhc I understood that FAB will be excluded... does it make sense to test this release then atm? Or shall I wait until provider release is cut?

jscheffl commented 1 week ago

Checked source tree of 2.10.3rc1 for fixes #42281 #42478 #42486 #42628 #42915 #43044 #43089 #43139 - and can confirm it is included.

dabla commented 1 week ago

Is it normal following fix isn't in 2.10.3? See issue about it.

jscheffl commented 6 days ago

Is it normal following fix isn't in 2.10.3? See issue about it.

@dabla The issue/fix were not marked as 2.10 relevant, therefore it seems they got out of sight. I assume it is a mistake. ANd a maintainer need to backport this to v2-10-test branch. SHall I take this... so that it goes at least in 2.10.4 if too late now?

dabla commented 6 days ago

Is it normal following fix isn't in 2.10.3? See issue about it.

@dabla The issue/fix were not marked as 2.10 relevant, therefore it seems they got out of sight. I assume it is a mistake. ANd a maintainer need to backport this to v2-10-test branch. SHall I take this... so that it goes at least in 2.10.4 if too late now?

It’s not a real issue, just a wrong logging issue. I’m willing to backport it to the 2-10 test branch as well if necessary…

jscheffl commented 6 days ago

I tested both 43566 and 43577 and can confirm these are in RC2.

Unfortunately I found a minor glitch with 43566 (Mapped Operator Try Selector) and raised another PR to fix also index 0... where the logic in JavaScript was bad. I am not sure whether this glitch should halt RC2, I assume it is bothering but rather low/medium.

potiuk commented 3 days ago

Checked that all changes are merged are in.

danxian-baiheng commented 2 days ago

Im facing the same error as @raphaelauv but with 2.10.2.

Python version: 3.11.6
Airflow version: 2.10.2
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask/app.py", line 2529, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask_appbuilder/security/decorators.py", line 137, in wraps
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask_appbuilder/views.py", line 550, in list
    widgets = self._list()
              ^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask_appbuilder/baseviews.py", line 1186, in _list
    form = self.search_form.refresh()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask_appbuilder/forms.py", line 327, in refresh
    form = self(obj=obj)
           ^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/form.py", line 209, in __call__
    return type.__call__(cls, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/flask_wtf/form.py", line 73, in __init__
    super().__init__(formdata=formdata, **kwargs)
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/form.py", line 281, in __init__
    super().__init__(self._unbound_fields, meta=meta_obj, prefix=prefix)
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/form.py", line 49, in __init__
    field = meta.bind_field(self, unbound_field, options)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/meta.py", line 28, in bind_field
    return unbound_field.bind(form=form, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/fields/core.py", line 387, in bind
    return self.field_class(*self.args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nucolumnaradmin/.venv/lib/python3.11/site-packages/wtforms/fields/core.py", line 133, in __init__
    for k, v in flags.items():
                ^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute 'items'

Is this resolved in the latest version, or it's caused by my fault? Currently we are using 2.6.3 in prod now, and we want to upgrade to a newer version(must >= 2.9.0). Which version will be recommended?

potiuk commented 2 days ago

This is because we were waiting for FAB provider 1.5.0 - it's been released today and we tested it before so we are just about to fix constraints for 2.10.3 to make sure it uses the latest fab provider.

potiuk commented 2 days ago

Which version will be recommended?

2.10.3 once released

potiuk commented 2 days ago

(it's always the latest version that is recommended BTW).

utkarsharma2 commented 2 days ago

We've just released Apache Airflow 2.10.3 🎉

📦 PyPI: https://pypi.org/project/apache-airflow/2.10.3/ 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.10.3/ 🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/2.10.3/release_notes.html 🐳 Docker Image: "docker pull apache/airflow:2.10.3" 🚏 Constraints: https://github.com/apache/airflow/tree/constraints-2.10.3

Thanks to all the contributors who made this possible.