search

newrelic / newrelic-airflow-plugin

Send airflow metrics to New Relic!
Apache License 2.0
25 stars 19 forks source link

Make the plugin compatible with Airflow >= "v2.6.0" #43

Open shuvaevv opened 10 months ago

shuvaevv commented 10 months ago

Closes: #40

  1. Fix the DummyStatsLogger import. This bug appeared because DummyStatsLogger was renamed to NoStatsLogger in Airflow v2.6.0. The import of the DummyStatsLogger is try-excepted, so the DummyStatsLogger class is None. It leads to the AttributeError: 'NoneType' object has no attribute 'incr'.

  2. Fix the unexpected keyword argument error after the "tag" argument was added to StatsLogger in Airflow v2.6.0

TimPansino commented 10 months ago

Hello,

This PR looks helpful, but unfortunately the CI pipeline for this project was canned by security during an incident related to Azure. I'll have to rewrite it in GitHub Actions before we can merge anything. I'll get back to this as soon as we have a new CI in place.

TimPansino commented 10 months ago

As of right now we don't have plans to finish fixing the CI, so this is unlikely to be merged or released. We're in talks of archiving this project as it's nowhere near as stable as the builtin OpenTelemetry offering.

If you'd like to give that a try, I'd highly recommend it over the current state of this plugin.

I put up a new documentation page about it here

thesuperzapper commented 2 weeks ago

@TimPansino I can confirm that this works in Airflow 2.9.3, can we please merge this?

Otherwise, people won't be able to use the newrelic-airflow-plugin with Airflow 2.6+

thesuperzapper commented 2 weeks ago

@TimPansino I also want to highlight that Airflow has a critical issue in its OpenTelemetry integrations, which will cause tasks to fail if the OpenTelemetry endpoint is down:

The NewRelic needs to fix this issue in airflow if you don't intend to support this plugin anymore.