search

logstash-plugins / logstash-output-s3

Apache License 2.0
57 stars 150 forks source link

Gzip compression not working #259

Open kunalmohan opened 1 year ago

kunalmohan commented 1 year ago

Logstash information:

Please include the following information:

  1. Logstash version - 
    logstash 8.8.1
  2. Logstash installation source - homebrew
  3. How is Logstash being run- via commandline
  4. How was the Logstash Plugin installed - no separate installation command

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version) - openjdk version "20.0.1" 2023-04-18
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc). - homebrew
  3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system): Darwin Mac.local 22.4.0 Darwin Kernel Version 22.4.0

Description of the problem including expected versus actual behavior: When specifying encoding => "gzip" in the config file, the file uploaded to S3 has .gz extension but is just a plain text file. I'm using "json_lines" codec.

ShaiBarak commented 3 months ago

I'm experiencing this as well, also when try to read with spark job i get an error : "java.io.IOException: not a gzip file" also using codec => "json_lines"