Closed BenDerPan closed 6 years ago
[ INFO ] *****************************************************
[ INFO ] ServerCount: 1
[ INFO ] 192.168.102.160:5304
[ INFO ] ProxyCount: 0
[ INFO ] *****************************************************
[ INFO ] Checking Client-Server protocol signatures...
[ DEBUG ] Preserved sync (rsp) message GetCellType is registered.
[ DEBUG ] Preserved sync (rsp) message QueryMemoryWorkingSet is registered.
[ DEBUG ] Preserved async message Shutdown is registered.
[ DEBUG ] Sync (rsp) message 0 is registered.
[ DEBUG ] Sync (rsp) message 1 is registered.
[ DEBUG ] Sync (rsp) message 2 is registered.
[ INFO ] Listening endpoint :5304
[ INFO ] Waiting for client connection ...
[ INFO ] My IPEndPoint: 127.0.1.1:5304
[ INFO ] *****************************************************
[ INFO ] ServerCount: 1
[ INFO ] 192.168.102.160:5304
[ INFO ] ProxyCount: 0
[ INFO ] *****************************************************
[ DEBUG ] ServerSocket: Incomming connection from 215.58.192.168
[ DEBUG ] ServerSocket: Incomming connection from 215.60.192.168
[ INFO ] Checking Server-Server protocol signatures...
[ DEBUG ] ServerSocket: Incomming connection from 215.68.192.168
[ DEBUG ] ServerSocket: Incomming connection from 215.70.192.168
uh oh, looks like the networking subsystem crashed on connection. investigating.
@BenDerPan hey could you try the eventloop branch? The Linux networking is improved.
I’ve also noticed the weird addresses reported by the server on client connection — these connections should be all coming from localhost, but appear to be random in the log.
@yatli great , I will try it later, not on my PC now. :)
@yatli the address report is still strange, but there is a new problem, my env : Server on ubuntu 16.04x64, Client on windows 10, and GraphEngine 2.0.9542.
The client side sometimes will dead with no response when I save someting, but sometime it's ok. I think it is still the problem of network
@BenDerPan thanks for testing! how about the double free corruption?
@yatli there is no exception on Linux server side, so the double free corruption seems fixed. but I am not sure.
attempting a minimal repro.
minimal repro failed.
Client side Windows 10 x64 eventloop HEAD
:
using System;
using Trinity;
using Trinity.Storage;
namespace test_ge
{
class Program
{
static void Main(string[] args)
{
TrinityConfig.LoadConfig("trinity.xml");
TrinityConfig.CurrentRunningMode = RunningMode.Client;
Global.CloudStorage.LoadCell(0, out var cell, out _);
Console.WriteLine(cell.Length);
}
}
}
Server side Linux, eventloop HEAD
:
using System;
using Trinity;
using Trinity.Storage;
using Trinity.Network;
namespace test
{
class Program
{
static void Main(string[] args)
{
Global.LocalStorage.SaveCell(0, new byte[128]);
Global.LocalStorage.SaveStorage();
TrinityServer server = new TrinityServer();
server.Start();
Console.Write("Press any key to stop...");
Console.ReadKey();
}
}
}
windows client correctly outputs 128
and exits.
@BenDerPan custom syn protocol?
@yatli yes, I used custom syn protocol
@BenDerPan repro failed.
Client:
using System;
using Trinity;
using Trinity.Storage;
using test_ge.S;
namespace test_ge
{
class Program
{
static void Main(string[] args)
{
TrinityConfig.LoadConfig("trinity.xml");
TrinityConfig.CurrentRunningMode = RunningMode.Client;
Global.CloudStorage.LoadCell(0, out var cell, out _);
Console.WriteLine(cell.Length);
using(var rsp = Global.CloudStorage[0].P())
{
Console.WriteLine(rsp);
}
}
}
}
server (Linux):
using System;
using Trinity;
using Trinity.Storage;
using Trinity.Network;
namespace test
{
class Server: SBase
{
public override void PHandler(PayloadWriter rsp)
{
rsp.foo = 123;
rsp.bar = "bar";
}
}
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
Global.LocalStorage.SaveCell(0, new byte[128]);
Global.LocalStorage.SaveStorage();
Server server = new Server();
server.Start();
Console.Write("Press any key to stop...");
Console.ReadKey();
}
}
}
TSL:
struct Payload
{
int foo;
string bar;
}
protocol P
{
Type: Syn;
Request: void;
Response: Payload;
}
server S
{
protocol P;
}
The client correctly outputs the response.
@yatli I am trying to clean all the cache ,and rebuild for a test with my project
@yatli Still the same: dead again, no response, as pic:
@BenDerPan you mean crash, exception or freeze?
@yatli I mean freeze, the code stopped at that line ,but there is no crash or exception
Understood. From your screenshot I see that I should try larger payloads.
Attempting a repro.
@BenDerPan do you observe the same symptom if running a windows server program?
@yatli I am trying now
@yatli not freeze, but there is exception:
在 Trinity.Storage.RemoteStorage._error_check(TrinityErrorCode err)
在 Trinity.Storage.RemoteStorage._use_synclient(Func`2 func)
在 Trinity.Storage.RemoteStorage.SendMessage(Byte* message, Int32 size, TrinityResponse& response)
在 Trinity.Storage.MessagePassingExtensionMethods.GetCommunicationSchema(IMessagePassingEndpoint storage, String& name, String& signature)
在 Trinity.Storage.MemoryCloud.CheckProtocolSignatures_impl(RemoteStorage storage, RunningMode from, RunningMode to)
在 Trinity.Storage.FixedMemoryCloud.Open(ClusterConfig config, Boolean nonblocking)
在 Trinity.Global.get_CloudStorage()
在
two possible cases:
could you come up with a minimal repro? I can then proceed to debug it.
@yatli sorry I can't , it's part of our big system, pick it out is a terrible work :(
@yatli I tested run the server on windows ,and build simple query client on both Ubuntu and windows, everything is ok , now I guess it's the version problem, server use eventloop version, and my exception client use master version.
@BenDerPan you mean you rebuild everything and the problem is gone?
@yatli yes, I can't reproduce the problem now, but I don't know why , maybe because I rebooted my PC.
alright. let's close this issue for now (as the original double-free corruption is known to be resolved). you're welcome to open up new issues following up this topic.
thanks again!
My Test Env:
GraphEngine.Core: V2.0.9328(latest) , I rebuild this morning.
OS: Ubuntu16.04 x64
.Net Core: V2.1.4
My code run well on windows, the exception occurs when I run my same code on Linux. and I found that when the storage contains data and query data with syn protocol this will happen.
My server side output
My client side only throw exception: System.IO.IOException "Network error occurs."