search

AdaCore / aws

AWS is a complete framework to develop Web based applications in Ada.
Other
131 stars 37 forks source link

Sometimes all the threads of the server fall and the server stops responding to requests #370

Closed khv2online closed 1 year ago

khv2online commented 1 year ago

Nginx (HTTP2) is used as a reverse proxy server.

Process logs

Fatal error, slot 5 is dead now.
raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x6bb1a5 Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x6bb615 Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x752463 Aws.Net.Send at aws-net.adb:480
0x66929e Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x5dad81 Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x7da122 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7d5500 Aws.Server.LineT at aws-server.adb:395
0x9d1b39 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fea3b8d7ea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fea3b6b3a2d
0xfffffffffffffffe
khv2online commented 1 year ago

This doesn't seem to happen again. So I'll close the PR

AJ-Ianozi commented 4 months ago

I'm running into this exact same problem with AWS 24 (alire), gnat 13.2.1 (alire). Server is running fine for hours and hours all day, but overnight I come back to this in my log; same files as @khv2online 's:

Fatal error, slot 5 is dead now.

raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x62eedc Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x62f31f Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x6bb3f4 Aws.Net.Send at aws-net.adb:480
0x5e4a67 Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x56282d Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x736b90 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7322ab Aws.Server.LineT at aws-server.adb:395
0x8ec059 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fbc4393eea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fbc43718a6d
0xfffffffffffffffe

Fatal error, slot 3 is dead now.

raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x62eedc Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x62f31f Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x6bb3f4 Aws.Net.Send at aws-net.adb:480
0x5e4a67 Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x56282d Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x736b90 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7322ab Aws.Server.LineT at aws-server.adb:395
0x8ec059 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fbc4393eea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fbc43718a6d
0xfffffffffffffffe

Fatal error, slot 2 is dead now.

raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x62eedc Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x62f31f Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x6bb3f4 Aws.Net.Send at aws-net.adb:480
0x5e4a67 Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x56282d Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x736b90 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7322ab Aws.Server.LineT at aws-server.adb:395
0x8ec059 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fbc4393eea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fbc43718a6d
0xfffffffffffffffe

Fatal error, slot 4 is dead now.

raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x62eedc Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x62f31f Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x6bb3f4 Aws.Net.Send at aws-net.adb:480
0x5e4a67 Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x56282d Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x736b90 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7322ab Aws.Server.LineT at aws-server.adb:395
0x8ec059 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fbc4393eea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fbc43718a6d
0xfffffffffffffffe

Fatal error, slot 1 is dead now.

raised AWS.NET.SOCKET_ERROR : Send : [32] Broken pipe
0x62eedc Aws.Net.Std.Raise_Exception at aws-net-std__gnat.adb:632
0x62f31f Aws.Net.Std.Send at aws-net-std__gnat.adb:695
0x6bb3f4 Aws.Net.Send at aws-net.adb:480
0x5e4a67 Aws.Net.Buffered.Flush at aws-net-buffered.adb:76
0x56282d Aws.Http2.Frame.Send at aws-http2-frame.adb:209
0x736b90 Aws.Server.Protocol_Handler at aws-server-protocol_handler.adb:295
0x7322ab Aws.Server.LineT at aws-server.adb:395
0x8ec059 System.Tasking.Stages.Task_Wrapper at s-tassta.adb:1211
[/lib/x86_64-linux-gnu/libpthread.so.0]
0x7fbc4393eea5
[/lib/x86_64-linux-gnu/libc.so.6]
0x7fbc43718a6d
0xfffffffffffffffe
TurboGit commented 4 months ago

@AJ-Ianozi : Would you be able to test a patch?

AJ-Ianozi commented 4 months ago

@TurboGit Absolutely.

TurboGit commented 4 months ago

Here it is, you can do a simple edit of the aws-server-protocol_handler.adb file based on the change below:

diff --git a/src/core/aws-server-protocol_handler.adb b/src/core/aws-server-protocol_handler.adb
index 5cb54e481..63d9fca73 100644
--- a/src/core/aws-server-protocol_handler.adb
+++ b/src/core/aws-server-protocol_handler.adb
@@ -290,9 +290,17 @@ begin
               or else (CNF.HTTP2_Activated (LA.Server.Properties)
                        and then not CNF.Security (LA.Server.Properties))
             then
-               HTTP2.Frame.GoAway.Create
-                 (Stream_Id => 0,
-                  Error     => HTTP2.C_Protocol_Error).Send (Sock_Ptr.all);
+               --  We don't want to fail, if the socket is closed already
+               --  we ignore the socket error.
+
+               begin
+                  HTTP2.Frame.GoAway.Create
+                    (Stream_Id => 0,
+                     Error     => HTTP2.C_Protocol_Error).Send (Sock_Ptr.all);
+               exception
+                  when Net.Socket_Error =>
+                     null;
+               end;

                exit For_Every_Request;
AJ-Ianozi commented 4 months ago

Thanks, I'll let you know how it goes!

AJ-Ianozi commented 3 months ago

@TurboGit Just wanted to follow back up, it's been a week and the server has not crashed yet. I think the patch is the solution.