[Bug] Diagnostics crash

[AuMilliat]

Hello,

When I activate diagnostics, the pipeline crash rapidly after RunAsynch call:

Context: 3 TCPSource Tuple<Vector3, Vector3> from Quest 2 headset with Unity.

Server code:

 public class PsiFormatPositionAndOrientation
    {
        public static Format<Tuple<System.Numerics.Vector3, System.Numerics.Vector3>> GetFormat()
        {
            return new Format<Tuple<System.Numerics.Vector3, System.Numerics.Vector3>>(WritePositionOrientation, ReadPositionOrientation);
        }

        public static void WritePositionOrientation(Tuple<System.Numerics.Vector3, System.Numerics.Vector3> point3D, BinaryWriter writer)
        {
            writer.Write((double)point3D.Item1.X);
            writer.Write((double)point3D.Item1.Y);
            writer.Write((double)point3D.Item1.Z);
            writer.Write((double)point3D.Item2.X);
            writer.Write((double)point3D.Item2.Y);
            writer.Write((double)point3D.Item2.Z);
        }

        public static Tuple<System.Numerics.Vector3, System.Numerics.Vector3> ReadPositionOrientation(BinaryReader reader)
        {
            return new Tuple<System.Numerics.Vector3, System.Numerics.Vector3>(new System.Numerics.Vector3((float)reader.ReadDouble(), (float)reader.ReadDouble(), (float)reader.ReadDouble()),
                            new System.Numerics.Vector3((float)reader.ReadDouble(), (float)reader.ReadDouble(), (float)reader.ReadDouble()));
        }
    }

    internal class Program
    {
        static private void Connection<T>(string name, Rendezvous.TcpSourceEndpoint? source, Pipeline p, Format<T> deserializer)
        {
            source?.ToTcpSource<T>(p, deserializer, null, true, name).Do((d, e) => { Console.WriteLine($"Recieve {name} data @{e} : {d}"); });
        }

        static void Quest2Demo(Pipeline p)
        {
            var host = "10.44.1.159";
            var remoteClock = new RemoteClockExporter(port: 11510);

             bool canStart = false;
            var process = new Rendezvous.Process("Server", new[] { remoteClock.ToRendezvousEndpoint(host)});
            var server = new RendezvousServer();
            server.Rendezvous.TryAddProcess(process);
            server.Rendezvous.ProcessAdded += (_, pr) =>
            {
                Console.WriteLine($"Process {pr.Name}");
                if (pr.Name == "Unity")
                {
                    foreach (var endpoint in pr.Endpoints)
                    {
                        if (endpoint is Rendezvous.TcpSourceEndpoint)
                        {
                            Rendezvous.TcpSourceEndpoint? source = endpoint as Rendezvous.TcpSourceEndpoint;
                            foreach (var stream in endpoint.Streams)
                            {
                                Console.WriteLine($"\tStream {stream.StreamName}");
                                switch (stream.StreamName)
                                {
                                    case "PositionLeft":
                                    case "PositionRight":
                                    case "Player":
                                        Connection<Tuple<System.Numerics.Vector3, System.Numerics.Vector3>>(stream.StreamName, source, p, PsiFormatPositionAndOrientation.GetFormat());
                                        break;

                                }
                            }
                        }
                        canStart = true;
                    }
                }
            };
            server.Error += (s, e) => { Console.WriteLine(e.Message); Console.WriteLine(e.HResult); };
            server.Start();
            while (!canStart) Thread.Sleep(500);
            Thread.Sleep(500);
        }
        static void Main(string[] args)
        {
            // Create the \psi pipeline
            Pipeline pipeline = Pipeline.Create("Subpipline Removal",enableDiagnostics:true);
            var store = PsiStore.Create(pipeline, "Diagnostics", $"F:/Stores/Diagnostics/");
            store.Write(pipeline.Diagnostics, "Pipeline");
            Quest2Demo(pipeline);

            // Start the pipeline running
            pipeline.RunAsync();
           // Waiting for an out key
           Console.WriteLine("Press any key to stop the application.");
           Console.ReadLine();
           // Stop correctly the pipeline.
           pipeline.Dispose();
       }
}

The only thing I can think of, is the RemoteClockImporter setting the pipeline VirtualTimeOffset while it's running... But it should trigger exception on the messages side (?)

[AuMilliat]

It might be the frequency of the data recieved, I've reduced it and the crash is not happening.

[sandrist]

Thanks for the details on this issue. Apparently, it is easy to overflow the Average() operator with too many DateTimes in the list. We will push a fix soon that should allow it to work at your original frequency.

[sandrist]

The fact that this TimeSpan values are so large is a bit suspicious though. What do the latencies look like on the streams you ship over from the Quest headset side? Do they look reasonable, or do they look really huge? There might be an issue in your message timestamps...

[sandrist]

Yeah, looking closer at those TimeSpan values, it appears that something is wrong with your message timestamps somewhere. They are showing spans of 584,388 days, which is roughly 1601 years. January 1, 1601 is the Windows epoch time. So I'm guessing you're not using the right UTC time somewhere, resulting in thousand-year latencies :)

[AuMilliat]

Thanks for the reply Sean, it seems a parsing/serialisation issue in DateTime, the years are not correct.

Here, the GetCurrentTime from pipeline in Unity side and the OriginatingTime in the server side:

[AuMilliat]

I've wrote a TCPWriter for DateTime in Unity and using the same method for serialization as Format class:

But it's not a Diagnostics crash issue :)