Are you ready to take your AWS RDS monitoring to the next level? Say hello to prometheus-rds-exporter, your ultimate solution for comprehensive, real-time insights into your Amazon RDS instances!
Built by SRE Engineers, designed for production: Meticulously crafted by a team of Site Reliability Engineers with years of hands-on experience in managing RDS production systems. Trust in their expertise to supercharge your monitoring.
It collects key metrics about:
[!TIP] Prometheus RDS exporter is part of the Database Monitoring Framework which provides alerts, along with their handy runbooks for AWS RDS.
π₯ Advanced Metrics: Gain deep visibility with advanced metrics for AWS RDS. Monitor performance, query efficiency, and resource utilization like never before.
𧩠AWS Quotas Insights: Stay in control with real-time information about AWS quotas. Ensure you never hit limits unexpectedly.
π‘ Hard Limits visibility: Know the hard limits of the EC2 instance used by RDS and manage your resources effectively.
π Alerting at Your Fingertips: Easily set up Prometheus alerting rules to stay informed of critical events, ensuring you're always ahead of issues.
π οΈ Simple Setup: Getting started is a breeze! Our clear documentation and examples will have you up and running in no time.
π Dashboards: Prometheus-RDS Exporter export adopts the USE methodology and provides well-designed, ready-to-use dashboards.
π Community-Driven: Join a vibrant community of users and contributors. Collaborate, share knowledge, and shape the future of AWS RDS monitoring together.
π When combined with prometheus-community/postgres_exporter, it provides a production-ready monitoring framework for RDS PostgreSQL.
Name | Labels | Description |
---|---|---|
rds_allocated_disk_iops_average | aws_account_id , aws_region , dbidentifier |
Allocated disk IOPS |
rds_allocated_disk_throughput_bytes | aws_account_id , aws_region , dbidentifier |
Allocated disk throughput |
rds_allocated_storage_bytes | aws_account_id , aws_region , dbidentifier |
Allocated storage |
rds_api_call_total | api , aws_account_id , aws_region |
Number of call to AWS API |
rds_backup_retention_period_seconds | aws_account_id , aws_region , dbidentifier |
Automatic DB snapshots retention period |
rds_ca_certificate_valid_until | aws_account_id , aws_region , dbidentifier |
Timestamp of the expiration of the Instance certificate |
rds_cpu_usage_percent_average | aws_account_id , aws_region , dbidentifier |
Instance CPU used |
rds_database_connections_average | aws_account_id , aws_region , dbidentifier |
The number of client network connections to the database instance |
rds_dbload_average | aws_account_id , aws_region , dbidentifier |
Number of active sessions for the DB engine |
rds_dbload_cpu_average | aws_account_id , aws_region , dbidentifier |
Number of active sessions where the wait event type is CPU |
rds_dbload_noncpu_average | aws_account_id , aws_region , dbidentifier |
Number of active sessions where the wait event type is not CPU |
rds_exporter_build_info | build_date , commit_sha , version |
A metric with constant '1' value labeled by version from which exporter was built |
rds_exporter_errors_total | Total number of errors encountered by the exporter | |
rds_free_storage_bytes | aws_account_id , aws_region , dbidentifier |
Free storage on the instance |
rds_freeable_memory_bytes | aws_account_id , aws_region , dbidentifier |
Amount of available random access memory. For MariaDB, MySQL, Oracle, and PostgreSQL DB instances, this metric reports the value of the MemAvailable field of /proc/meminfo |
rds_instance_age_seconds | aws_account_id , aws_region , dbidentifier |
Time since instance creation |
rds_instance_baseline_iops_average | aws_account_id , aws_region , instance_class |
Baseline IOPS of underlying EC2 instance class |
rds_instance_baseline_throughput_bytes | aws_account_id , aws_region , instance_class |
Baseline throughput of underlying EC2 instance class |
rds_instance_info | arn , aws_account_id , aws_region , dbi_resource_id , dbidentifier , deletion_protection , engine , engine_version , instance_class , multi_az , performance_insights_enabled , pending_maintenance , pending_modified_values , role , source_dbidentifier , storage_type , ca_certificate_identifier |
RDS instance information |
rds_instance_log_files_size_bytes | aws_account_id , aws_region , dbidentifier |
Total of log files on the instance |
rds_instance_max_iops_average | aws_account_id , aws_region , instance_class |
Maximum IOPS of underlying EC2 instance class |
rds_instance_max_throughput_bytes | aws_account_id , aws_region , instance_class |
Maximum throughput of underlying EC2 instance class |
rds_instance_memory_bytes | aws_account_id , aws_region , instance_class |
Instance class memory |
rds_instance_status | aws_account_id , aws_region , dbidentifier |
Instance status (1: ok, 0: can't scrap metrics) |
rds_instance_tags | aws_account_id , aws_region , dbidentifier , tag_<AWS_TAG> ... |
AWS tags attached to the instance |
rds_instance_vcpu_average | aws_account_id , aws_region , instance_class |
Total vCPU for this instance class |
rds_max_allocated_storage_bytes | aws_account_id , aws_region , dbidentifier |
Upper limit in gibibytes to which Amazon RDS can automatically scale the storage of the DB instance |
rds_max_disk_iops_average | aws_account_id , aws_region , dbidentifier |
Max disk IOPS evaluated with disk IOPS and EC2 capacity |
rds_max_storage_throughput_bytes | aws_account_id , aws_region , dbidentifier |
Max disk throughput evaluated with disk throughput and EC2 capacity |
rds_maximum_used_transaction_ids_average | aws_account_id , aws_region , dbidentifier |
Maximum transaction IDs that have been used. Applies to only PostgreSQL |
rds_quota_max_dbinstances_average | aws_account_id , aws_region |
Maximum number of RDS instances allowed in the AWS account |
rds_quota_maximum_db_instance_snapshots_average | aws_account_id , aws_region |
Maximum number of manual DB instance snapshots |
rds_quota_total_storage_bytes | aws_account_id , aws_region |
Maximum total storage for all DB instances |
rds_read_iops_average | aws_account_id , aws_region , dbidentifier |
Average number of disk read I/O operations per second |
rds_read_throughput_bytes | aws_account_id , aws_region , dbidentifier |
Average number of bytes read from disk per second |
rds_replica_lag_seconds | aws_account_id , aws_region , dbidentifier |
For read replica configurations, the amount of time a read replica DB instance lags behind the source DB instance. Applies to MariaDB, Microsoft SQL Server, MySQL, Oracle, and PostgreSQL read replicas |
rds_replication_slot_disk_usage_bytes | aws_account_id , aws_region , dbidentifier |
Disk space used by replication slot files. Applies to PostgreSQL |
rds_swap_usage_bytes | aws_account_id , aws_region , dbidentifier |
Amount of swap space used on the DB instance. This metric is not available for SQL Server |
rds_transaction_logs_disk_usage_bytes | aws_account_id , aws_region , dbidentifier |
Disk space used by transaction logs (only on PostgreSQL) |
rds_usage_allocated_storage_bytes | aws_account_id , aws_region |
Total storage used by AWS RDS instances |
rds_usage_db_instances_average | aws_account_id , aws_region |
AWS RDS instance count |
rds_usage_manual_snapshots_average | aws_account_id , aws_region |
Manual snapshots count |
rds_write_iops_average | aws_account_id , aws_region , dbidentifier |
Average number of disk write I/O operations per second |
rds_write_throughput_bytes | aws_account_id , aws_region , dbidentifier |
Average number of bytes written to disk per second |
up | Was the last scrape of RDS successful |
[!TIP] If you deploy Grafana operator in your Kubernetes cluster, dashboards could be automatically deployed and maintained up-to-date. Set
dashboards.enabled: true
in your Helm deployment to deploy dashboards as GrafanaDashboard CRD
For convenience, dashboards are also available in configs/grafana/public/
folder and Grafana labs:
![Instances overview](docs/screenshots/instances-overview.png) RDS instances JSON or 19647 | ![Instance details](docs/screenshots/instance-details.png) RDS instance details JSON or 19646 | ![RDS exporters](docs/screenshots/rds-exporter.png) Prometheus RDS exporter JSON or 19679 |
Configuration could be defined in prometheus-rds-exporter.yaml or environment variables (format PROMETHEUS_RDS_EXPORTER_<PARAMETER_NAME>
).
Parameter | Description | Default |
---|---|---|
aws-assume-role-arn | AWS IAM ARN role to assume to fetch metrics | |
aws-assume-role-session | AWS assume role session name | prometheus-rds-exporter |
collect-instance-metrics | Collect AWS instances metrics (AWS Cloudwatch API) | true |
collect-instance-tags | Collect AWS RDS tags | true |
collect-instance-types | Collect AWS instance types information (AWS EC2 API) | true |
collect-logs-size | Collect AWS instances logs size (AWS RDS API) | true |
collect-maintenances | Collect AWS instances maintenances (AWS RDS API) | true |
collect-quotas | Collect AWS RDS quotas (AWS quotas API) | true |
collect-usages | Collect AWS RDS usages (AWS Cloudwatch API) | true |
debug | Enable debug mode | |
enable-otel-traces | Enable OpenTelemetry traces. See configuration | false |
listen-address | Address to listen on for web interface | :9043 |
log-format | Log format (text or json ) |
json |
metrics-path | Path under which to expose metrics | /metrics |
tls-cert-path | Path to TLS certificate | |
tls-key-path | Path to private key for TLS |
Configuration parameters priorities:
$HOME/prometheus-rds-exporter.yaml
fileprometheus-rds-exporter.yaml
filePrometheus RDS exporter needs read-only AWS IAM permissions to fetch metrics from AWS RDS, CloudWatch, EC2 and ServiceQuota AWS APIs.
Standard AWS authentication methods (AWS credentials, SSO and assumed role) are supported, see https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html.
If you are running on AWS EKS, we strongly recommend to use IRSA
We recommend deployment using helm Helm.
See all available configuration parameters in configs/helm/values.yaml
See the Development environment to start the Prometheus RDS exporter, Prometheus, and Grafana with dashboards in a minute.
Recommended method to deploy on AWS EKS using IRSA and Helm.
Create an IAM policy
IAM_POLICY_NAME=prometheus-rds-exporter
# Download policy payload
curl --fail --silent --write-out "Reponse code: %{response_code}\n" https://raw.githubusercontent.com/qonto/prometheus-rds-exporter/main/configs/aws/policy.json -o /tmp/prometheus-rds-exporter.policy.json
# Create IAM policy
aws iam create-policy --policy-name ${IAM_POLICY_NAME} --policy-document file:///tmp/prometheus-rds-exporter.policy.json
Create and attach an IAM role to your EKS cluster
eksctl will create an IAM role and a Kubernetes Service account
EKS_CLUSTER_NAME=default # Replace with your EKS cluster name
KUBERNETES_NAMESPACE=monitoring # Replace with namespace of your choice
IAM_ROLE_NAME=prometheus-rds-exporter
KUBERNETES_SERVICE_ACCOUNT_NAME=prometheus-rds-exporter
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
eksctl \
create iamserviceaccount \
--cluster ${EKS_CLUSTER_NAME} \
--namespace ${KUBERNETES_NAMESPACE} \
--name ${KUBERNETES_SERVICE_ACCOUNT_NAME} \
--role-name ${IAM_ROLE_NAME} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/${IAM_POLICY_NAME} \
--approve
Deploy the exporter
PROMETHEUS_RDS_EXPORTER_VERSION=0.3.0 # Replace with latest version
SERVICE_ACCOUNT_ANNOTATION="arn:aws:iam::${AWS_ACCOUNT_ID}:role/${IAM_ROLE_NAME}"
helm upgrade \
prometheus-rds-exporter \
oci://public.ecr.aws/qonto/prometheus-rds-exporter-chart \
--version ${PROMETHEUS_RDS_EXPORTER_VERSION} \
--install \
--namespace ${KUBERNETES_NAMESPACE} \
--set serviceAccount.annotations."eks\.amazonaws\.com\/role-arn"="${SERVICE_ACCOUNT_ANNOTATION}"
Option. Customize Prometheus exporter settings
Download Helm chart default values
helm show values oci://public.ecr.aws/qonto/prometheus-rds-exporter-chart --version ${PROMETHEUS_RDS_EXPORTER_VERSION} > values.yaml
Customize settings
vim values.yaml
Update Helm deployment:
helm upgrade \
prometheus-rds-exporter \
oci://public.ecr.aws/qonto/prometheus-rds-exporter-chart \
--version ${PROMETHEUS_RDS_EXPORTER_VERSION} \
--install \
--namespace ${KUBERNETES_NAMESPACE} \
--set serviceAccount.annotations."eks\.amazonaws\.com\/role-arn"="${SERVICE_ACCOUNT_ANNOTATION}" \
--values values.yaml
Grant IAM permissions to the EC2 instance
Download the Debian package
PROMETHEUS_RDS_EXPORTER_VERSION=0.3.0 # Replace with latest version
PACKAGE_NAME=prometheus-rds-exporter_${PROMETHEUS_RDS_EXPORTER_VERSION}_$(uname -m).deb
wget https://github.com/qonto/prometheus-rds-exporter/releases/download/${PROMETHEUS_RDS_EXPORTER_VERSION}/${PACKAGE_NAME}
Install package
Prometheus RDS exporter will be automatically started as a service.
dpkg -i ${PACKAGE_NAME}
Optional, customize configuration
# Copy configuration template
cp /usr/share/prometheus-rds-exporter/prometheus-rds-exporter.yaml.sample /var/lib/prometheus-rds-exporter/prometheus-rds-exporter.yaml
# Edit configuration
vim /var/lib/prometheus-rds-exporter/prometheus-rds-exporter.yaml
# Restart service
systemctl restart prometheus-rds-exporter
Binary
PROMETHEUS_RDS_EXPORTER_VERSION=0.3.0 # Replace with latest version
TARBALL_NAME=prometheus-rds-exporter_Linux_$(uname -m).tar.gz
wget https://github.com/qonto/prometheus-rds-exporter/releases/download/${PROMETHEUS_RDS_EXPORTER_VERSION}/${TARBALL_NAME}
tar xvzf ${TARBALL_NAME}
Optional, customize configuration
vim prometheus-rds-exporter.yaml
Start the exporter
./prometheus-rds-exporter
Connect on AWS with any method
aws configure
Start application
docker run -p 9043:9043 -e AWS_PROFILE=${AWS_PROFILE} -v $HOME/.aws:/app/.aws public.ecr.aws/qonto/prometheus-rds-exporter:latest
percona/rds_exporter and mtanda/rds_enhanced_monitoring_exporter are great alternatives.
prometheus/cloudwatch_exporter could be used to collect additional CloudWatch metrics.
See CONTRIBUTING.md.
To report a security issue, please visit SECURITY.md
You can start a simple development environment using the Docker compose configuration in /scripts/prometheus
.
It will start Grafana (with the dashboards), Prometheus, and the RDS exporter:
Connect on AWS using the AWS CLI
Launch development stack
cd scripts/prometheus
docker compose up --build
Connect on the services
Execute Go tests:
make test
Execute Helm chart tests:
make helm-test # Helm unit test
make kubeconform # Kubernetes manifest validation
make checkcov # Check misconfigurations
Prometheus RDS Exporter includes an OpenTelemetry trace exporter to facilitate troubleshooting.
Traces can be forwarded to any OpenTelemetry server using gRPC protocol.
Export the OTEL_EXPORTER_OTLP_ENDPOINT
variable.
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
See OTEL SDK configuration and OpenTelemetry environments variables for all options.
Start exporter with OpenTelemetry enabled
prometheus-rds-exporter --enable-otel-traces