search

GoogleCloudPlatform / oozie-to-airflow

Oozie Workflow to Airflow DAGs migration tool
Apache License 2.0
87 stars 50 forks source link

chore(deps): bump apache-airflow from 1.10.14 to 2.7.1 #701

Closed dependabot[bot] closed 1 year ago

dependabot[bot] commented 1 year ago

Bumps apache-airflow from 1.10.14 to 2.7.1.

Release notes

Sourced from apache-airflow's releases.

Apache Airflow 2.7.1

Significant Changes

CronTriggerTimetable is now less aggressive when trying to skip a run (#33404)

When setting catchup=False, CronTriggerTimetable no longer skips a run if the scheduler does not query the timetable immediately after the previous run has been triggered.

This should not affect scheduling in most cases, but can change the behaviour if a DAG is paused-unpaused to manually skip a run. Previously, the timetable (with catchup=False) would only start a run after a DAG is unpaused, but with this change, the scheduler would try to look at little bit back to schedule the previous run that covers a part of the period when the DAG was paused. This means you will need to keep a DAG paused longer (namely, for the entire cron period to pass) to really skip a run.

Note that this is also the behaviour exhibited by various other cron-based scheduling tools, such as anacron.

conf.set() becomes case insensitive to match conf.get() behavior (#33452)

Also, conf.get() will now break if used with non-string parameters.

conf.set(section, key, value) used to be case sensitive, i.e. conf.set("SECTION", "KEY", value) and conf.set("section", "key", value) were stored as two distinct configurations. This was inconsistent with the behavior of conf.get(section, key), which was always converting the section and key to lower case.

As a result, configuration options set with upper case characters in the section or key were unreachable. That's why we are now converting section and key to lower case in conf.set too.

We also changed a bit the behavior of conf.get(). It used to allow objects that are not strings in the section or key. Doing this will now result in an exception. For instance, conf.get("section", 123) needs to be replaced with conf.get("section", "123").

Bug Fixes

  • Ensure that tasks wait for running indirect setup (#33903)
  • Respect "soft_fail" for core async sensors (#33403)
  • Differentiate 0 and unset as a default param values (#33965)
  • Raise 404 from Variable PATCH API if variable is not found (#33885)
  • Fix MappedTaskGroup tasks not respecting upstream dependency (#33732)
  • Add limit 1 if required first value from query result (#33672)
  • Fix UI DAG counts including deleted DAGs (#33778)
  • Fix cleaning zombie RESTARTING tasks (#33706)
  • SECURITY_MANAGER_CLASS should be a reference to class, not a string (#33690)
  • Add back get_url_for_login in security manager (#33660)
  • Fix 2.7.0 db migration job errors (#33652)
  • Set context inside templates (#33645)
  • Treat dag-defined access_control as authoritative if defined (#33632)
  • Bind engine before attempting to drop archive tables (#33622)
  • Add a fallback in case no first name and last name are set (#33617)

... (truncated)

Changelog

Sourced from apache-airflow's changelog.

Airflow 2.7.1 (2023-09-07)

Significant Changes ^^^^^^^^^^^^^^^^^^^

CronTriggerTimetable is now less aggressive when trying to skip a run (#33404) """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

When setting catchup=False, CronTriggerTimetable no longer skips a run if the scheduler does not query the timetable immediately after the previous run has been triggered.

This should not affect scheduling in most cases, but can change the behaviour if a DAG is paused-unpaused to manually skip a run. Previously, the timetable (with catchup=False) would only start a run after a DAG is unpaused, but with this change, the scheduler would try to look at little bit back to schedule the previous run that covers a part of the period when the DAG was paused. This means you will need to keep a DAG paused longer (namely, for the entire cron period to pass) to really skip a run.

Note that this is also the behaviour exhibited by various other cron-based scheduling tools, such as anacron.

conf.set() becomes case insensitive to match conf.get() behavior (#33452) """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""

Also, conf.get() will now break if used with non-string parameters.

conf.set(section, key, value) used to be case sensitive, i.e. conf.set("SECTION", "KEY", value) and conf.set("section", "key", value) were stored as two distinct configurations. This was inconsistent with the behavior of conf.get(section, key), which was always converting the section and key to lower case.

As a result, configuration options set with upper case characters in the section or key were unreachable. That's why we are now converting section and key to lower case in conf.set too.

We also changed a bit the behavior of conf.get(). It used to allow objects that are not strings in the section or key. Doing this will now result in an exception. For instance, conf.get("section", 123) needs to be replaced with conf.get("section", "123").

Bug Fixes """""""""

  • Ensure that tasks wait for running indirect setup (#33903)
  • Respect "soft_fail" for core async sensors (#33403)
  • Differentiate 0 and unset as a default param values (#33965)
  • Raise 404 from Variable PATCH API if variable is not found (#33885)
  • Fix MappedTaskGroup tasks not respecting upstream dependency (#33732)
  • Add limit 1 if required first value from query result (#33672)
  • Fix UI DAG counts including deleted DAGs (#33778)
  • Fix cleaning zombie RESTARTING tasks (#33706)
  • SECURITY_MANAGER_CLASS should be a reference to class, not a string (#33690)

... (truncated)

Commits


Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/GoogleCloudPlatform/oozie-to-airflow/network/alerts).
dependabot[bot] commented 1 year ago

Looks like apache-airflow is up-to-date now, so this is no longer needed.