LitJson reading from a network stream blocks even when a complete object is read

[logicle]

When LitJson creates a JsonReader from a NetworkStream, it will continue to attempt to read even after a complete Json object has been parsed.

This read will block and not return the Json object.

The block happens in the lexer (which is correct, it should if it is in the middle of parsing an object until the rest of the object arrives on the stream):

        private int NextChar ()
        {
            if (input_buffer != 0) {
                int tmp = input_buffer;
                input_buffer = 0;

                return tmp;
            }

            return reader.Read ();
        }

Here is block of code that demonstrates the bug:

            NetworkStream stream = tcpClient.GetStream();
            while (keepReading)
            {
                keepReading = (null != tcpClient) && tcpClient.Connected;
                if (keepReading)
                {
                    System.IO.TextReader tr = new System.IO.StreamReader(stream);
                    JsonReader jsonReader = new JsonReader(tr);

                    // this will block on a network stream
                    JsonData result = JsonMapper.ToObject(jsonReader);

                    // with the bug, this line is never reached
                    System.Console.WriteLine(result.ToJson());
                }
            }

The expected behavior is that JsonMapper.ToObject (and all variants), will return when a complete Json object has been parsed from the stream. If the stream has fragments of another message not yet complete, JsonMapper.ToObject should first return the complete message, then in the next iteration, block on the stream until the rest of the next message arrives and return that object, etc...

Here is a complete c# program that can be used to reproduce and debug the problem:

using System;
using System.Threading;
using System.Net;
using System.Net.Sockets;
using LitJson;

// pulled from another project. The threading model is preserved
// in case it is at all relevant to reproducing the bug.
namespace LitJsonNetworkTest
{
    public class ServerConnection
    {
        private TcpClient tcpClient = null;
        private Thread networkReadThread = null;

        public ServerConnection()
        {
        }

        public void Wait()
        {
            networkReadThread.Join();
        }
        public void ConnectTo(string a, int p)
        {
            tcpClient = new TcpClient(a, p);
            networkReadThread = new Thread(NetworkReadThread);
            networkReadThread.Start();
        }

        void NetworkReadThread()
        {
            bool keepReading = (null != tcpClient);
            NetworkStream stream = tcpClient.GetStream();
            while (keepReading)
            {
                keepReading = (null != tcpClient) && tcpClient.Connected;
                try
                {
                    if (keepReading)
                    {
                        System.IO.TextReader tr = new System.IO.StreamReader(stream);
                        JsonReader jsonReader = new JsonReader(tr);

                        // this will block on a network stream
                        JsonData result = JsonMapper.ToObject(jsonReader);

                        // with the bug, this line is never reached
                        System.Console.WriteLine(result.ToJson());
                    }
                }
                catch (System.Exception e)
                {
                    string errorMessage = "Connection exception: " + e.Message + "\nStack Trace " + e.StackTrace;
                    string stackTrace = e.StackTrace;
                    System.Console.WriteLine(errorMessage);
                    throw (e);
                }
            }
        }
    }

    class Session
    {
        private TcpClient m_tcpClient = null;
        private Thread m_workerThread = null;

        public Session(TcpClient client)
        {
            if (null != client)
            {
                m_tcpClient = client;
                m_workerThread = new Thread(this.Start);
                m_workerThread.Start();
            }
            else if (null != client)
            {
                m_tcpClient = client;
                Shutdown();
            }
        }

        private void Wait()
        {
            m_workerThread.Join();
        }

        void Send(string payload)
        {
            byte[] responseBytes = System.Text.Encoding.ASCII.GetBytes(payload);
            m_tcpClient.GetStream().Write(responseBytes, 0, payload.Length);
        }

        void Shutdown()
        {
            m_tcpClient.Close();
        }

        private void Start()
        {
            try
            {
                Send("{\"playerId\":\"logicle@live.com\",\"randomData\":\"dooblydoo\"}");
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                Console.WriteLine(e.StackTrace);
                Shutdown();
                return;
            }
        }
    }

    class Server
    {
        private IPAddress m_listenAddress = IPAddress.Parse("0.0.0.0");
        private TcpListener m_server = null;
        private Thread m_workerThread = null;

        public Server()
        {
            m_server = new TcpListener(m_listenAddress, 42424);
            m_workerThread = new Thread(this.Start);
            m_workerThread.Start();
        }

        public void wait()
        {
            m_workerThread.Join();
        }

        private void Start()
        {
            try
            {
                m_server.Start(64);
                while(true)
                {
                    TcpClient client = m_server.AcceptTcpClient();
                    if(null != client)
                    {
                        Session s = new Session(client);
                    }
                }
            }
            catch(Exception e)
            {
                Console.WriteLine(e.Message);
                Console.WriteLine(e.StackTrace);
            }
        }
    }
    class Program
    {
        static void Main(string[] args)
        {
            Server s = new Server();

            ServerConnection server = new ServerConnection();
            server.ConnectTo("127.0.0.1", 42424);
            server.Wait();
            s.wait();
        }
    }
}

I would not be surprised it this is actually user error, but this seems like a reasonable usage of passing a TextStream to JsonWrapper/JsonReader.

I will dig through the source to see if there is a sane way to support the expected behavior.

[logicle]

Still digging.

[joethephish]

I believe I'm having a similar issue. I'd like to have two blocks of JSON stored within the same file. The first is read by LitJson, but then I need to use an API that uses its own built-in JSON reader.

I agree - it seems like if streams are supported at all (rather than pure strings), then LitJson ought to leave the stream in the right state.

Here's a simple example of what I'm trying to do:

// Pass in two pieces of concatenated JSON to a StringReader
var stringReader = new StringReader("{}{}");

// Try to read the first JSON object
var obj = LitJson.JsonMapper.ToObject(stringReader);

// Now we want the second block: "{}"
// But this returns "}" since LitJson has already stepped beyond
// what it's supposed to be interested in.
var secondJsonObjText = stringReader.ReadToEnd();