I configured my ALB logs and also ELB logs write to a s3, and configured graylog-plugin-s3 to read logs from s3 but only ALB logs are streamed from s3 not ELB logs because ELB logs are not in gzip format. By default ELB logs will not be written in gzip https://forums.aws.amazon.com/thread.jspa?threadID=244860 it will in plain text format.
2018-08-01T13:01:47.416Z ERROR [CloudTrailSubscriber] Could not read CloudTrail log file for <app-aws-logs>. Skipping.
com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input in field name
at [Source: java.io.StringReader@9760ebb; line: 1, column: 10026237]
at [Source: java.io.StringReader@9760ebb; line: 1, column: 9999952] (through reference chain: org.graylog.aws.inputs.cloudtrail.json.CloudTrailRecordList["Records"]->java.util.Arra
yList[6168]->org.graylog.aws.inputs.cloudtrail.json.CloudTrailRecord["userIdentity"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:388) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:348) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapAndThrow(BeanDeserializerBase.java:1600) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:278) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:140) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:287) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:259) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:504) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:111) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:276) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:140) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3814) ~[graylog.jar:?]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2858) ~[graylog.jar:?]
at org.graylog.aws.inputs.cloudtrail.messages.TreeReader.read(TreeReader.java:19) ~[graylog-plugin-aws-2.4.6.jar:?]
at org.graylog.aws.inputs.cloudtrail.CloudTrailSubscriber.run(CloudTrailSubscriber.java:108) [graylog-plugin-aws-2.4.6.jar:?]
Caused by: com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input in field name
extension is .log in case of elb
What is the workaround for this ?
I configured my ALB logs and also ELB logs write to a s3, and configured graylog-plugin-s3 to read logs from s3 but only ALB logs are streamed from s3 not ELB logs because ELB logs are not in gzip format. By default ELB logs will not be written in gzip
https://forums.aws.amazon.com/thread.jspa?threadID=244860it will in plain text format.extension is .log in case of elb What is the workaround for this ?