search

bluedenim / log4j-s3-search

Log4j appender with S3, Azure, Google Cloud, and search publishing
MIT License
65 stars 44 forks source link

AWS S3 - lack of data compression #104

Closed Suqu13 closed 3 years ago

Suqu13 commented 3 years ago

I noticed that the s3Compression flag is not used within appender-log4j2. I suppose that it is omitted in the given method. Using config.setCompressionEnabled(s3Compression) will solve the problem.

I suggest also to add additional flag that cooperates with s3Comporession to give users choice to decide if they want to use .gz file extension or not. A lot of AWS resources, like Athena, work well with compressed files but their names have to contain the suffix.

bluedenim commented 3 years ago

Thanks. I'll take a look.

bluedenim commented 3 years ago

Release 3.1.2 fixes the bug in Log4j2 appender builder to recognize the s3Compress property. It was missing a line before.

I'll start 3.2.0 to add an optional property to append a .gz to the file extension when using compression

bluedenim commented 3 years ago

Release 3.2.0 now also includes a new configuration key for S3 (s3KeyGzSuffixEnabled) to append .gz to the key name.