cloudtrail-parquet-glue
is terraform module that builds a Glue workflow to convert CloudTrail S3 logs to Athena-friendly Parquet format and make them available as a table using a Glue Crawler.
CloudTrailRawCrawler
) to build the raw CloudTrail logs table with partitionsCloudTrailToParquet
) to convert that table to Parquet formatCloudTrailParquetCrawler
) to build the CloudTrail logs table with partitionsetl_script_s3_bucket
), one used for temporary processing (temp_s3_bucket
), and one used to store the parquet files (parquet_s3_bucket
).module "cloudtrail_parquet_glue" {
source = "github.com/alsmola/cloudtrail_parquet_glue"
region = "us-east-1"
aws_account_id "1234567890"
cloudtrail_s3_bucket = "s3://my-cloudtrail-bucket"
etl_script_s3_bucket = "s3://my-etl-script-bucket"
parquet_s3_bucket = "s3://my-parquet-bucket"
temp_s3_bucket = = "s3://my-temp-bucket"
glue_database = "cloudtrail"
}
raw_[your_cloudtrailbucket_name]
and cloudtrail_parquet
)