search

aws / aws-sdk

Landing page for the AWS SDKs on GitHub
https://aws.amazon.com/tools/
Other
68 stars 13 forks source link

RetentionInDays not set when describing a cloudwatch log group via the LogGroupNamePrefix #608

Closed sean-kuendig-copebit closed 9 months ago

sean-kuendig-copebit commented 1 year ago

Describe the bug

The CloudWatchLogs client provides two options for retrieving a log group by its name: LogGroupNamePattern and LogGroupNamePrefix. When using the LogGroupNamePrefix parameter, the RetentionInDays field of the log group is set. However, if you use the LogGroupNamePattern parameter instead, the RetentionInDays field is not set.

Expected Behavior

The field RetentionInDays should be consistently set, regardless if the log group is being described by its name with the parameter LogGroupNamePattern or LogGroupNamePrefix.

Current Behavior

Only the log group that is described via LogGroupNamePrefix has the RetentionInDays set. If the log group is described by using LogGroupNamePattern the RetentionInDays is nil.

The output below happens when the code in the reproduction steps is run:

Prefix Group:
bslbgm
30
Pattern Group:
bslbgm
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x131adde]

Reproduction Steps

package main

import (
    "context"
    "fmt"
    "strings"
    "time"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs"
    "github.com/gruntwork-io/terratest/modules/random"
)

func main() {
    ctx := context.Background()
    cfg, _ := config.LoadDefaultConfig(ctx)
    cloudwatchlogsClient := cloudwatchlogs.NewFromConfig(cfg)

    logGroupNameSpace := strings.ToLower(random.UniqueId())

    // A new log group needs to be created so that the lambda is triggered
    cloudwatchlogsClient.CreateLogGroup(ctx, &cloudwatchlogs.CreateLogGroupInput{
        LogGroupName: &logGroupNameSpace,
    })

    time.Sleep(time.Second * time.Duration(10))

    cloudwatchlogsClient.PutRetentionPolicy(ctx, &cloudwatchlogs.PutRetentionPolicyInput{
        LogGroupName:    &logGroupNameSpace,
        RetentionInDays: aws.Int32(30),
    })

    time.Sleep(time.Second * time.Duration(10))

    prefix_output, _ := cloudwatchlogsClient.DescribeLogGroups(ctx, &cloudwatchlogs.DescribeLogGroupsInput{
        LogGroupNamePrefix: &logGroupNameSpace,
    })

    pattern_output, _ := cloudwatchlogsClient.DescribeLogGroups(ctx, &cloudwatchlogs.DescribeLogGroupsInput{
        LogGroupNamePattern: &logGroupNameSpace,
    })

    fmt.Println("Prefix Group:")
    for _, prefix_group := range prefix_output.LogGroups {
        fmt.Println(*prefix_group.LogGroupName)
        fmt.Println(*prefix_group.RetentionInDays)
    }

    fmt.Println("Pattern Group:")
    for _, pattern_group := range pattern_output.LogGroups {
        fmt.Println(*pattern_group.LogGroupName)
        // This is where it fails because the retention in days is not set
        fmt.Println(*pattern_group.RetentionInDays)
    }
}

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/jmespath/go-jmespath@v0.4.0 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/feature/ec2/imds@v1.13.2 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/internal/ini@v1.3.33 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/config@v1.18.21 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2/feature/ec2/imds@v1.13.2 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/credentials@v1.13.20 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2/feature/ec2/imds@v1.13.2 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/feature/ec2/imds@v1.13.2 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/feature/ec2/imds@v1.13.2 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2/internal/configsources@v1.1.32 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2/internal/ini@v1.3.33 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@v1.20.9 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@v1.20.9 github.com/aws/aws-sdk-go-v2/internal/configsources@v1.1.32 github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@v1.20.9 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs@v1.20.9 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url@v1.9.26 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url@v1.9.26 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url@v1.9.26 github.com/google/go-cmp@v0.5.8 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/aws-sdk-go-v2/internal/configsources@v1.1.32 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/aws-sdk-go-v2/service/sso@v1.12.8 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/aws-sdk-go-v2/internal/configsources@v1.1.32 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/aws-sdk-go-v2/service/ssooidc@v1.14.8 github.com/aws/smithy-go@v1.13.5 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2@v1.17.8 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2/internal/configsources@v1.1.32 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2/internal/endpoints/v2@v2.4.26 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/aws-sdk-go-v2/service/internal/presigned-url@v1.9.26 github.com/aws/aws-sdk-go-v2/service/sts@v1.18.9 github.com/aws/smithy-go@v1.13.5 github.com/aws/smithy-go@v1.13.5 github.com/google/go-cmp@v0.5.8 github.com/aws/smithy-go@v1.13.5 github.com/jmespath/go-jmespath@v0.4.0

Compiler and Version used

go version go1.19.3 darwin/amd64

Operating System and version

MacOs Ventura 13.0.1

jmklix commented 1 year ago

The default is indefinite retentions so it looks like this is just not being changed when you are trying to set it to 30. Looking into this.

jmklix commented 9 months ago

Sorry for the delay. I was able to reproduce this and it seems to be that the problem is with the service not this sdk. The response for logGroupNamePattern is missing the retentionInDays

logGroupNamePrefix: 

{"logGroups":[{"arn":"arn:aws:logs:us-west-2:xxxxxxxxxx:log-group:mfb2sj:*","creationTime":1696373299319,"logGroupName":"mfb2sj","metricFilterCount":0,"retentionInDays":30,"storedBytes":0}]}

logGroupNamePattern:

{"logGroups":[{"arn":"arn:aws:logs:us-west-2:xxxxxxxxxx:log-group:mfb2sj:*","creationTime":1696373299319,"logGroupName":"mfb2sj","metricFilterCount":0}],"nextToken":"nexttokenstring"}

Opening internal request to service team

jmklix commented 9 months ago

P101732296

jmklix commented 9 months ago

This behaving as expected. As noted in the docs here:

If you specify logGroupNamePattern in your request, then only arn, creationTime, and logGroupName are included in the response.

So with logGroupNamePattern you can't get the RetentionInDays. Please let me know if you have any other questions

github-actions[bot] commented 9 months ago

This issue is now closed.

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.