Open vinitsriwastava opened 1 year ago
@vinitsriwastava Any update from your side? Facing the same issue at the moment. Here is my setup
airflow.cfg
[metrics]
statsd_on = True
statsd_host = localhost
statsd_port = 8125
statsd_prefix = airflow
statsd_allow_list = scheduler
requirements.txt
apache-airflow==2.3.4
newrelic-airflow-plugin==0.2.0
newrelic-telemetry-sdk==0.4.3
statsd==3.3.0
systemd serivce file
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service
Wants=postgresql.service mysql.service
[Service]
EnvironmentFile=/etc/environment
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/bin/bash -c 'source {{ venv }}/{{ hostname }}/bin/activate && NEW_RELIC_CONFIG_FILE=/etc/systemd/system/newrelic-airflow-scheduler.ini newrelic-admin run-program {{ venv }}/{{ hostname }}/bin/airflow scheduler'
Restart=on-failure
RestartSec=5s
PrivateTmp=true
[Install]
WantedBy=multi-user.target
/etc/environment
NEW_RELIC_INSERT_KEY="redacted"
NEW_RELIC_SERVICE_NAME="airflow-stats-dev"
NEW_RELIC_HOST="metric-api.eu.newrelic.com"
Hi @a-feld @TimPansino ,
I have an airflow cluster on azure where webserver, scheduler and workers are running on VM as sytemctl service, install newrelic airflow plugin on all VM's but metrics is not getting recorded in newrelic. Note: statsd install and pre-requisites done in airflow config.
In log I just see this {newrelic_plugin.py:126} INFO - Using NewRelicStatsLogger
The next info log for recording is not executing i.e. line 66 (-- Using New Relic Stats Recorder)
Can you please let me know what can be the probable reason and any directions to resolve this issue?
Thanks Sumit