awslabs / aws-embedded-metrics-python

Amazon CloudWatch Embedded Metric Format Client Library
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format.html
Apache License 2.0
202 stars 34 forks source link

Using create_metrics_logger() directly? #71

Open jennakwon06 opened 3 years ago

jennakwon06 commented 3 years ago

Hello!

I'm wondering what happens if I use create_metrics_logger() directly to create a MetricsLogger and put a metric.

The problem I am trying to solve is below.

Within my lambda, I have a function below that I call thousands of times to validate thousands of data points, and log violation if validation fails.

    @metric_scope
    def apply_validation_and_log_violation(
        self,
        value: Any,
        default_value: Any,
        validation_func: Callable,
        metrics: MetricsLogger
    ):
        if validation_func(value):
            return value

        metrics.put_metric("ValidationFailed", 1, "Count")

But calling a function, wrapped with @metric_scope decorator, thousands of times is flooding my cloudwatch log. It seems like for each function call, below is getting logged:

{"LogGroup": "test", "ServiceName": "test", "ServiceType": "AWS::Lambda::Function", "executionEnvironment": "AWS_Lambda_python3.8", "memorySize": "512", "functionVersion": "$LATEST", "logStreamId": "2021/09/09/[$LATEST]83e2421e4e5dbf0e84c58c4d0876b3fd", "traceId": "Root=1-dc99d00f-c079a84d433534434534ef0d;Parent=91ed514f1e5c03b2;Sampled=1", "_aws": {"Timestamp": 1631212956097, "CloudWatchMetrics": [{"Dimensions": [["LogGroup", "ServiceName", "ServiceType"]], "Metrics": [], "Namespace": "aws-embedded-metrics"}]}}

So I am wondering if I can create a MetricsLogger object with create_metrics_logger function and pass that object around.

Thanks!

benkehoe commented 2 years ago

I'm a little confused. Are you decorating your Lambda function handler with @metric_scope, and invoking the Lambda function many times, or decorating a function that you call from your Lambda function handler? If it's the latter, I'd recommend moving @metric_scope to decorate your Lambda function handler, and passing the same MetricsLogger into this function.

It does make sense to me that create_metrics_logger should be exposed in the top-level package.