jckuester / awsls

A list command for AWS resources
MIT License
830 stars 53 forks source link

Occasional garbled json from scans #60

Open puluanau opened 7 months ago

puluanau commented 7 months ago

First of all, thanks for the great tool, and especially the insight to plug together existing components in a new ways, that's even more sexy. Just started using this a few days ago, just reporting in case others also have seen.

I am scanning accounts mostly just forking a bunch of individual awsls instances to get the json, an example cmd line here is:

 awsls -r ap-south-1,eu-north-1,eu-west-3,eu-west-2,eu-west-1,ap-northeast-3,ap-northeast-2,ap-northeast-1,ca-central-1,sa-east-1,ap-southeast-1,ap-southeast-2,eu-central-1,us-east-1,us-east-2,us-west-1,us-west-2, --json --attributes tags {aws_iam_policy,aws_redshift_subnet_group,aws_vpc_endpoint,aws_ecrpublic_repository,aws_nat_gateway,aws_cloudwatch_log_resource_policy,aws_ebs_volume,aws_guardduty_detector,aws_route_table,aws_iam_user,aws_redshift_parameter_group,aws_redshift_cluster,aws_iam_role,aws_key_pair,aws_mwaa_environment} > MY_JSON.json

Each of these instances sends its output to its own file. However, since the app itself is multithreaded there are rare occasions where the output from two queries gets spit out together, and the resulting json is then invalid.

No idea how to fix it, not begging for free help (I just regenerate the failed ones for now), but just FYI. Maybe a mutex on stdout? https://forum.golangbridge.org/t/thread-safe-operation-with-stdout-and-stderr/23949 .