nessadc
commented
1 year ago We're also seeing this behavior.
Logs
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | timestamp | message | |---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 1680031182901 | INIT_START Runtime Version: java:8.v17 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:xxxx | | 1680031183466 | Transforming org/apache/logging/log4j/core/lookup/JndiLookup (java.net.URLClassLoader@37f8bb67) | | 1680031186391 | START RequestId: 3026ec16-737d-475f-8175-3057e93c81a7 Version: $LATEST | | 1680031186448 | 2023-03-28 19:19:46 3026ec16-737d-475f-8175-3057e93c81a7 INFO GraylogS3Function:65 - Log level is now set to [INFO]. | | 1680031186448 | io.netty.channel.socket.nio.NioSocketChannel | | 1680031186450 | org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder | | 1680031186450 | io.netty.channel.MultithreadEventLoopGroup | | 1680031186450 | io.netty.util.internal.InternalThreadLocalMap | | 1680031186450 | io.netty.util.internal.PlatformDependent0 | | 1680031186451 | io.netty.channel.AbstractChannel | | 1680031186451 | io.netty.util.concurrent.AbstractEventExecutor | | 1680031186451 | io.netty.util.concurrent.GlobalEventExecutor | | 1680031186451 | org.graylog2.gelfclient.transport.AbstractGelfTransport | | 1680031186452 | io.netty.util.internal.PlatformDependent | | 1680031186452 | org.graylog2.gelfclient.transport.GelfSenderThread | | 1680031186452 | com.github.joschi.jadconfig.JadConfig | | 1680031186452 | io.netty.channel.AbstractChannelHandlerContext | | 1680031186453 | io.netty.channel.nio.AbstractNioChannel | | 1680031186453 | org.graylog.integrations.s3.CompressionType | | 1680031186453 | io.netty.channel.DefaultChannelId | | 1680031186453 | io.netty.channel.ChannelInitializer | | 1680031186454 | io.netty.util.concurrent.SingleThreadEventExecutor | | 1680031186454 | org.graylog.integrations.s3.codec.CloudflareLogCodec | | 1680031186454 | io.netty.util.NetUtilInitializations | | 1680031186455 | org.graylog2.gelfclient.transport.GelfTcpTransport | | 1680031186455 | io.netty.util.NetUtil | | 1680031186455 | io.netty.util.concurrent.DefaultPromise.rejectedExecution | | 1680031186455 | io.netty.util.internal.CleanerJava6 | | 1680031186456 | io.netty.util.ResourceLeakDetector | | 1680031186456 | io.netty.buffer.PooledByteBufAllocator | | 1680031186456 | io.netty.util.internal.MacAddressUtil | | 1680031186456 | io.netty.util.internal.SystemPropertyUtil | | 1680031186457 | io.netty.channel.ChannelOutboundBuffer | | 1680031186457 | io.netty.util.concurrent.DefaultPromise | | 1680031186457 | io.netty.bootstrap.Bootstrap | | 1680031186457 | io.netty.resolver.AddressResolverGroup | | 1680031186458 | io.netty.buffer.ByteBufUtil | | 1680031186458 | io.netty.buffer.PoolThreadCache | | 1680031186458 | org.graylog.integrations.s3.S3EventProcessor | | 1680031186458 | io.netty.channel.nio.NioEventLoop | | 1680031186458 | org.graylog.integrations.s3.S3ScannerFactory | | 1680031186459 | org.graylog.integrations.s3.GraylogS3Function | | 1680031186459 | org.graylog.integrations.s3.ProtocolType | | 1680031186459 | org.graylog.integrations.s3.ContentType | | 1680031186459 | io.netty.util.internal.logging.InternalLoggerFactory | | 1680031186460 | io.netty.channel.DefaultChannelPipeline | | 1680031186460 | io.netty.channel.ChannelHandlerMask | | 1680031186461 | 2023-03-28 19:19:46 3026ec16-737d-475f-8175-3057e93c81a7 INFO S3EventProcessor:45 - Reading object [xxxx.log.gz] from bucket [logsbucket] | | 1680031187194 | 2023-03-28 19:19:47 3026ec16-737d-475f-8175-3057e93c81a7 INFO S3EventProcessor:94 - Sent [66] messages. | | 1680031187498 | 2023-03-28 19:19:47 INFO GelfTcpTransport:115 - Channel disconnected! | | 1680031189510 | 2023-03-28 19:19:49 3026ec16-737d-475f-8175-3057e93c81a7 INFO GraylogS3Function:42 - Processed [1] S3 events. | | 1680031189519 | END RequestId: 3026ec16-737d-475f-8175-3057e93c81a7 | | 1680031189519 | REPORT RequestId: 3026ec16-737d-475f-8175-3057e93c81a7 Duration: 3127.83 ms Billed Duration: 3128 ms Memory Size: 4096 MB Max Memory Used: 204 MB Init Duration: 3482.92 ms | | 1680031194242 | START RequestId: 3d738d28-507e-4f40-9e7e-8493629a6297 Version: $LATEST | | 1680031194244 | 2023-03-28 19:19:54 3d738d28-507e-4f40-9e7e-8493629a6297 INFO GraylogS3Function:65 - Log level is now set to [INFO]. | | 1680031194247 | 2023-03-28 19:19:54 3d738d28-507e-4f40-9e7e-8493629a6297 INFO S3EventProcessor:45 - Reading object [xxxx.log.gz] from bucket [logsbucket] | | 1680031194247 | io.netty.channel.socket.nio.NioSocketChannel | | 1680031194247 | org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder | | 1680031194247 | io.netty.channel.MultithreadEventLoopGroup | | 1680031194247 | io.netty.util.internal.InternalThreadLocalMap | | 1680031194247 | io.netty.util.internal.PlatformDependent0 | | 1680031194247 | io.netty.channel.AbstractChannel | | 1680031194247 | io.netty.util.concurrent.AbstractEventExecutor | | 1680031194247 | io.netty.util.concurrent.GlobalEventExecutor | | 1680031194247 | org.graylog2.gelfclient.transport.AbstractGelfTransport | | 1680031194247 | io.netty.util.internal.PlatformDependent | | 1680031194247 | org.graylog2.gelfclient.transport.GelfSenderThread | | 1680031194247 | com.github.joschi.jadconfig.JadConfig | | 1680031194247 | io.netty.channel.AbstractChannelHandlerContext | | 1680031194247 | io.netty.channel.nio.AbstractNioChannel | | 1680031194247 | org.graylog.integrations.s3.CompressionType | | 1680031194247 | io.netty.channel.DefaultChannelId | | 1680031194247 | io.netty.buffer.AbstractByteBuf | | 1680031194247 | io.netty.channel.ChannelInitializer | | 1680031194247 | io.netty.util.concurrent.SingleThreadEventExecutor | | 1680031194247 | org.graylog.integrations.s3.codec.CloudflareLogCodec | | 1680031194247 | io.netty.util.NetUtilInitializations | | 1680031194247 | org.graylog2.gelfclient.transport.GelfTcpTransport | | 1680031194247 | io.netty.util.NetUtil | | 1680031194247 | io.netty.util.concurrent.DefaultPromise.rejectedExecution | | 1680031194247 | io.netty.util.internal.CleanerJava6 | | 1680031194247 | io.netty.util.ResourceLeakDetector | | 1680031194247 | io.netty.buffer.PooledByteBufAllocator | | 1680031194247 | io.netty.util.internal.MacAddressUtil | | 1680031194247 | io.netty.util.internal.SystemPropertyUtil | | 1680031194247 | io.netty.util.Recycler | | 1680031194247 | io.netty.channel.ChannelOutboundBuffer | | 1680031194247 | io.netty.util.concurrent.DefaultPromise | | 1680031194247 | io.netty.bootstrap.Bootstrap | | 1680031194247 | io.netty.resolver.AddressResolverGroup | | 1680031194247 | io.netty.buffer.ByteBufUtil | | 1680031194247 | io.netty.buffer.PoolThreadCache | | 1680031194247 | org.graylog.integrations.s3.S3EventProcessor | | 1680031194247 | io.netty.util.ReferenceCountUtil | | 1680031194247 | io.netty.channel.nio.NioEventLoop | | 1680031194247 | org.graylog.integrations.s3.S3ScannerFactory | | 1680031194247 | io.netty.util.ResourceLeakDetectorFactory | | 1680031194247 | org.graylog.integrations.s3.GraylogS3Function | | 1680031194247 | org.graylog.integrations.s3.ProtocolType | | 1680031194247 | org.graylog.integrations.s3.ContentType | | 1680031194247 | io.netty.util.internal.logging.InternalLoggerFactory | | 1680031194247 | io.netty.channel.DefaultChannelPipeline | | 1680031194247 | io.netty.channel.ChannelHandlerMask | | 1680031194394 | 2023-03-28 19:19:54 3d738d28-507e-4f40-9e7e-8493629a6297 INFO S3EventProcessor:94 - Sent [33] messages. | | 1680031374850 | 2023-03-28 19:22:54 3d738d28-507e-4f40-9e7e-8493629a6297 ERROR GelfSenderThread:175 - Failed to flush messages in [600] attempts. Shutting down anyway. | | 1680031374851 | 2023-03-28 19:22:54 3d738d28-507e-4f40-9e7e-8493629a6297 INFO GraylogS3Function:42 - Processed [1] S3 events. | | 1680031374852 | END RequestId: 3d738d28-507e-4f40-9e7e-8493629a6297 | | 1680031374852 | REPORT RequestId: 3d738d28-507e-4f40-9e7e-8493629a6297 Duration: 180608.48 ms Billed Duration: 180609 ms Memory Size: 4096 MB Max Memory Used: 204 MB | | 1680031379228 | START RequestId: 45d0ec0f-b71f-4e61-8826-675a6e80f9cb Version: $LATEST | | 1680031379230 | 2023-03-28 19:22:59 45d0ec0f-b71f-4e61-8826-675a6e80f9cb INFO GraylogS3Function:65 - Log level is now set to [INFO]. | | 1680031379230 | io.netty.channel.socket.nio.NioSocketChannel | | 1680031379230 | org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder | | 1680031379230 | io.netty.channel.MultithreadEventLoopGroup | | 1680031379230 | io.netty.util.internal.InternalThreadLocalMap | | 1680031379230 | io.netty.util.internal.PlatformDependent0 | | 1680031379230 | io.netty.channel.AbstractChannel | | 1680031379230 | io.netty.util.concurrent.AbstractEventExecutor | | 1680031379230 | io.netty.util.concurrent.GlobalEventExecutor | | 1680031379230 | org.graylog2.gelfclient.transport.AbstractGelfTransport | | 1680031379230 | io.netty.util.internal.PlatformDependent | | 1680031379230 | org.graylog2.gelfclient.transport.GelfSenderThread | | 1680031379230 | com.github.joschi.jadconfig.JadConfig | | 1680031379230 | io.netty.channel.AbstractChannelHandlerContext | | 1680031379230 | io.netty.channel.nio.AbstractNioChannel | | 1680031379231 | org.graylog.integrations.s3.CompressionType | | 1680031379231 | io.netty.channel.DefaultChannelId | | 1680031379231 | io.netty.buffer.AbstractByteBuf | | 1680031379231 | io.netty.channel.ChannelInitializer | | 1680031379231 | io.netty.util.concurrent.SingleThreadEventExecutor | | 1680031379231 | org.graylog.integrations.s3.codec.CloudflareLogCodec | | 1680031379231 | io.netty.util.NetUtilInitializations | | 1680031379231 | org.graylog2.gelfclient.transport.GelfTcpTransport | | 1680031379231 | io.netty.util.NetUtil | | 1680031379231 | io.netty.util.concurrent.DefaultPromise.rejectedExecution | | 1680031379231 | io.netty.util.internal.CleanerJava6 | | 1680031379231 | io.netty.util.ResourceLeakDetector | | 1680031379231 | io.netty.buffer.PooledByteBufAllocator | | 1680031379231 | io.netty.util.internal.MacAddressUtil | | 1680031379231 | io.netty.util.internal.SystemPropertyUtil | | 1680031379231 | io.netty.util.Recycler | | 1680031379231 | io.netty.channel.ChannelOutboundBuffer | | 1680031379231 | io.netty.util.concurrent.DefaultPromise | | 1680031379231 | io.netty.bootstrap.Bootstrap | | 1680031379231 | io.netty.resolver.AddressResolverGroup | | 1680031379231 | io.netty.buffer.ByteBufUtil | | 1680031379231 | io.netty.buffer.PoolThreadCache | | 1680031379231 | org.graylog.integrations.s3.S3EventProcessor | | 1680031379231 | io.netty.util.ReferenceCountUtil | | 1680031379232 | 2023-03-28 19:22:59 45d0ec0f-b71f-4e61-8826-675a6e80f9cb INFO S3EventProcessor:45 - Reading object [xxxx.log.gz] from bucket [logsbucket] | | 1680031379234 | io.netty.channel.nio.NioEventLoop | | 1680031379234 | org.graylog.integrations.s3.S3ScannerFactory | | 1680031379234 | io.netty.util.ResourceLeakDetectorFactory | | 1680031379234 | org.graylog.integrations.s3.GraylogS3Function | | 1680031379234 | org.graylog.integrations.s3.ProtocolType | | 1680031379234 | org.graylog.integrations.s3.ContentType | | 1680031379234 | io.netty.util.internal.logging.InternalLoggerFactory | | 1680031379234 | io.netty.channel.DefaultChannelPipeline | | 1680031379234 | io.netty.channel.ChannelHandlerMask | | 1680031379341 | 2023-03-28 19:22:59 45d0ec0f-b71f-4e61-8826-675a6e80f9cb INFO S3EventProcessor:94 - Sent [1] messages. | | 1680031559778 | 2023-03-28 19:25:59 45d0ec0f-b71f-4e61-8826-675a6e80f9cb ERROR GelfSenderThread:175 - Failed to flush messages in [600] attempts. Shutting down anyway. | | 1680031559778 | 2023-03-28 19:25:59 45d0ec0f-b71f-4e61-8826-675a6e80f9cb INFO GraylogS3Function:42 - Processed [1] S3 events. | | 1680031559779 | END RequestId: 45d0ec0f-b71f-4e61-8826-675a6e80f9cb | | 1680031559779 | REPORT RequestId: 45d0ec0f-b71f-4e61-8826-675a6e80f9cb Duration: 180549.62 ms Billed Duration: 180550 ms Memory Size: 4096 MB Max Memory Used: 204 MB | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
We are encountering an issue where log messages are not being sent to Graylog. When the lambda first starts, it's able to process one message but it fails for all subsequent runs without any apparent reason (or log message).
Runtime: Java 8 on Amazon Linux 2 Memory: 512 MB
I've tried tweaking the following but nothing seems to help
Logs
message INIT_START Runtime Version: java:8.al2.v18 Runtime Version ARN: arn:aws:lambda:us-west-2::runtime:xxx Transforming org/apache/logging/log4j/core/lookup/JndiLookup (lambdainternal.CustomerClassLoader@37bba400) START RequestId: 90958b19-01b9-4744-863f-1bde7ffc5c7a Version: $LATEST 2023-03-14 06:45:00 90958b19-01b9-4744-863f-1bde7ffc5c7a INFO GraylogS3Function:65 - Log level is now set to [DEBUG]. "io.netty.channel.socket.nio.NioSocketChannel " org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder "io.netty.channel.MultithreadEventLoopGroup " "io.netty.util.internal.InternalThreadLocalMap " io.netty.util.internal.PlatformDependent0 "io.netty.channel.AbstractChannel " "io.netty.util.concurrent.AbstractEventExecutor " "io.netty.util.concurrent.GlobalEventExecutor " org.graylog2.gelfclient.transport.AbstractGelfTransport "io.netty.util.internal.PlatformDependent " org.graylog2.gelfclient.transport.GelfSenderThread "com.github.joschi.jadconfig.JadConfig " "io.netty.channel.AbstractChannelHandlerContext " "io.netty.channel.nio.AbstractNioChannel " org.graylog.integrations.s3.CompressionType "io.netty.channel.DefaultChannelId " "io.netty.channel.ChannelInitializer " "io.netty.util.concurrent.SingleThreadEventExecutor " "io.netty.util.NetUtilInitializations " org.graylog2.gelfclient.transport.GelfTcpTransport "io.netty.util.NetUtil " "io.netty.util.concurrent.DefaultPromise.rejectedExecution " io.netty.util.internal.CleanerJava6 "io.netty.util.ResourceLeakDetector " "io.netty.buffer.PooledByteBufAllocator " "io.netty.util.internal.MacAddressUtil " "io.netty.util.internal.SystemPropertyUtil " "io.netty.channel.ChannelOutboundBuffer " "io.netty.util.concurrent.DefaultPromise " "io.netty.bootstrap.Bootstrap " "io.netty.resolver.AddressResolverGroup " "io.netty.buffer.ByteBufUtil " "io.netty.buffer.PoolThreadCache " org.graylog.integrations.s3.S3EventProcessor "io.netty.channel.nio.NioEventLoop " org.graylog.integrations.s3.S3ScannerFactory org.graylog.integrations.s3.GraylogS3Function org.graylog.integrations.s3.ProtocolType org.graylog.integrations.s3.ContentType "io.netty.util.internal.logging.InternalLoggerFactory " "io.netty.channel.DefaultChannelPipeline " "io.netty.channel.ChannelHandlerMask " "2023-03-14 06:45:00 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GraylogS3Function:37 - Configuration{, graylogHost='xxx.xx.xx.xxx', graylogPort=12201, connectTimeout=10000, reconnectDelay=10000, tcpKeepAlive=true, tcpNoDelay=true, queueSize=4, maxInflightSends=512, contentType='text/plain', compressionType='none', protocolType='tcp', messageFields='[]', messageSummaryFields='[ClientRequestHost, ClientRequestPath, OriginIP, ClientSrcPort, EdgeServerIP, EdgeResponseBytes]'}" 2023-03-14 06:45:00 90958b19-01b9-4744-863f-1bde7ffc5c7a INFO S3EventProcessor:45 - Reading object [prod-company-integration-webhooks-functionF19B1A04-8zoCVNRl0Zsn/2023/03/14/06-44-54-814351.log] from bucket [integration-webhooks-logs] "2023-03-14 06:45:03 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG AbstractGelfTransport:94 - Sending message: GelfMessage{version=""1.1"" timestamp=""1678776303.922"" short_message=""[{'time': '2023-03-14T06:43:54.697Z', 'type': 'platform.start', 'record': {'requestId': 'ad89fcd8-d804-4270-bb6b-b5e816c57a22', 'version': '8'}}, {'time': '2023-03-14T06:43:54.697Z', 'type': 'function', 'record': '{""request_id"": null, ""levelname"": ""INFO"", ""name"": ""company.webhook_router.handler"", ""module"": ""handler"", ""lineno"": 97, ""message"": ""Health check PASSED"", ""timestamp"": ""2023-03-14T06:43:54.697837+00:00""} '}, {'time': '2023-03-14T06:43:55.028Z', 'type': 'platform.end', 'record': {'requestId': 'ad89fcd8-d804-4270-bb6b-b5e816c57a22'}}, {'time': '2023-03-14T06:43:55.028Z', 'type': 'platform.report', 'record': {'requestId': 'ad89fcd8-d804-4270-bb6b-b5e816c57a22', 'metrics': {'durationMs': 331.874, 'billedDurationMs': 332, 'memorySizeMB': 128, 'maxMemoryUsedMB': 113}}}]"", level=""ALERT(1)""}" 2023-03-14 06:45:03 90958b19-01b9-4744-863f-1bde7ffc5c7a INFO S3EventProcessor:94 - Sent [1] messages. 2023-03-14 06:45:03 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG S3EventProcessor:51 - Waiting up to [100ms] with [10] retries while waiting for transport shutdown to occur. 2023-03-14 06:45:03 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:156 - Attempting to flush messages in [100/MILLISECONDS] with [10] retries "2023-03-14 06:45:03 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:04 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:164 - Flushing in progress. [0] messages are still enqueued, and [1] messages are still in-flight." "2023-03-14 06:45:04 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:164 - Flushing in progress. [0] messages are still enqueued, and [1] messages are still in-flight." "2023-03-14 06:45:04 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:164 - Flushing in progress. [0] messages are still enqueued, and [1] messages are still in-flight." 2023-03-14 06:45:04 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG GelfSenderThread:160 - Successfully flushed messages. Shutting down now. 2023-03-14 06:45:04 INFO GelfTcpTransport:115 - Channel disconnected! 2023-03-14 06:45:04 DEBUG GelfSenderThread:123 - GelfSenderThread exiting! 2023-03-14 06:45:06 DEBUG PoolThreadCache:224 - Freed 1 thread-local buffer(s) from thread: gelfTcpTransport-1-1 2023-03-14 06:45:06 90958b19-01b9-4744-863f-1bde7ffc5c7a DEBUG S3EventProcessor:55 - Transport shutdown complete. 2023-03-14 06:45:06 90958b19-01b9-4744-863f-1bde7ffc5c7a INFO GraylogS3Function:42 - Processed [1] S3 events. END RequestId: 90958b19-01b9-4744-863f-1bde7ffc5c7a REPORT RequestId: 90958b19-01b9-4744-863f-1bde7ffc5c7a Duration: 6418.68 ms Billed Duration: 6419 ms Memory Size: 512 MB Max Memory Used: 116 MB START RequestId: d6e85b8b-e151-45c3-826c-eb5c1a5938c3 Version: $LATEST "io.netty.channel.socket.nio.NioSocketChannel " org.graylog2.gelfclient.encoder.GelfMessageJsonEncoder "io.netty.channel.MultithreadEventLoopGroup " "io.netty.util.internal.InternalThreadLocalMap " io.netty.util.internal.PlatformDependent0 "io.netty.channel.AbstractChannel " "io.netty.util.concurrent.AbstractEventExecutor " "io.netty.util.concurrent.GlobalEventExecutor " org.graylog2.gelfclient.transport.AbstractGelfTransport "io.netty.util.internal.PlatformDependent " org.graylog2.gelfclient.transport.GelfSenderThread "com.github.joschi.jadconfig.JadConfig " "io.netty.channel.AbstractChannelHandlerContext " "io.netty.channel.nio.AbstractNioChannel " org.graylog.integrations.s3.CompressionType "io.netty.channel.DefaultChannelId " "io.netty.buffer.AbstractByteBuf " "io.netty.channel.ChannelInitializer " "io.netty.util.concurrent.SingleThreadEventExecutor " "io.netty.util.NetUtilInitializations " org.graylog2.gelfclient.transport.GelfTcpTransport "io.netty.util.NetUtil " "io.netty.util.concurrent.DefaultPromise.rejectedExecution " io.netty.util.internal.CleanerJava6 "io.netty.util.ResourceLeakDetector " "io.netty.buffer.PooledByteBufAllocator " "io.netty.util.internal.MacAddressUtil " "io.netty.util.internal.SystemPropertyUtil " "io.netty.util.Recycler " "io.netty.channel.ChannelOutboundBuffer " "io.netty.util.concurrent.DefaultPromise " "io.netty.bootstrap.Bootstrap " "io.netty.resolver.AddressResolverGroup " "io.netty.buffer.ByteBufUtil " "io.netty.buffer.PoolThreadCache " org.graylog.integrations.s3.S3EventProcessor "io.netty.util.ReferenceCountUtil " "io.netty.channel.nio.NioEventLoop " org.graylog.integrations.s3.S3ScannerFactory "io.netty.util.ResourceLeakDetectorFactory " org.graylog.integrations.s3.GraylogS3Function org.graylog.integrations.s3.ProtocolType org.graylog.integrations.s3.ContentType "io.netty.util.internal.logging.InternalLoggerFactory " "io.netty.channel.DefaultChannelPipeline " "io.netty.channel.ChannelHandlerMask " 2023-03-14 06:45:55 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 INFO GraylogS3Function:65 - Log level is now set to [DEBUG]. "2023-03-14 06:45:55 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GraylogS3Function:37 - Configuration{, graylogHost='xxx.xx.xx.xxx', graylogPort=12201, connectTimeout=10000, reconnectDelay=10000, tcpKeepAlive=true, tcpNoDelay=true, queueSize=4, maxInflightSends=512, contentType='text/plain', compressionType='none', protocolType='tcp', messageFields='[]', messageSummaryFields='[ClientRequestHost, ClientRequestPath, OriginIP, ClientSrcPort, EdgeServerIP, EdgeResponseBytes]'}" 2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 INFO S3EventProcessor:45 - Reading object [prod-company-integration-webhooks-functionF19B1A04-8zoCVNRl0Zsn/2023/03/14/06-45-54-692438.log] from bucket [integration-webhooks-logs] "2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG AbstractGelfTransport:94 - Sending message: GelfMessage{version=""1.1"" timestamp=""1678776356.731"" short_message=""[{'time': '2023-03-14T06:44:54.813Z', 'type': 'platform.start', 'record': {'requestId': 'a80f2222-c2fd-411d-9ac0-324ed2af8997', 'version': '8'}}, {'time': '2023-03-14T06:44:54.828Z', 'type': 'function', 'record': '{""request_id"": null, ""levelname"": ""INFO"", ""name"": ""company.webhook_router.handler"", ""module"": ""handler"", ""lineno"": 97, ""message"": ""Health check PASSED"", ""timestamp"": ""2023-03-14T06:44:54.814409+00:00""} '}, {'time': '2023-03-14T06:44:55.098Z', 'type': 'platform.end', 'record': {'requestId': 'a80f2222-c2fd-411d-9ac0-324ed2af8997'}}, {'time': '2023-03-14T06:44:55.098Z', 'type': 'platform.report', 'record': {'requestId': 'a80f2222-c2fd-411d-9ac0-324ed2af8997', 'metrics': {'durationMs': 284.761, 'billedDurationMs': 285, 'memorySizeMB': 128, 'maxMemoryUsedMB': 113}}}]"", level=""ALERT(1)""}" 2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 INFO S3EventProcessor:94 - Sent [1] messages. 2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG S3EventProcessor:51 - Waiting up to [100ms] with [10] retries while waiting for transport shutdown to occur. 2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:156 - Attempting to flush messages in [100/MILLISECONDS] with [10] retries "2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:56 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." "2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG GelfSenderThread:164 - Flushing in progress. [1] messages are still enqueued, and [0] messages are still in-flight." 2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 ERROR GelfSenderThread:175 - Failed to flush messages in [10] attempts. Shutting down anyway. 2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 DEBUG S3EventProcessor:55 - Transport shutdown complete. 2023-03-14 06:45:57 d6e85b8b-e151-45c3-826c-eb5c1a5938c3 INFO GraylogS3Function:42 - Processed [1] S3 events. END RequestId: d6e85b8b-e151-45c3-826c-eb5c1a5938c3 REPORT RequestId: d6e85b8b-e151-45c3-826c-eb5c1a5938c3 Duration: 1882.47 ms Billed Duration: 1883 ms Memory Size: 512 MB Max Memory Used: 116 MB