Flushing buffer to closed connection causes crash

[malekva]

When the connection with the database server is interrupted, flushing buffers to the sender cause crash. Example:

bool sendRandomBuffer(questdb::ilp::line_sender * sender)
{
  try  
  {
      questdb::ilp::line_sender_buffer buffer;

      auto bufferSize = randomInt();
      for(int i = 0; i < bufferSize; i++)
      {
          // CREATE TABLE myTable (myInt INT, myString STRING, time TIMESTAMP) TIMESTAMP(time);
          buffer 
              .table("myTable")
              .column("myInt", randomInt())
              .column("myString", randomString())
              .at_now();
      }
      sender->flush(buffer);
      return true;
  }
  catch(std::runtime_error & e)
  {
      std::cerr << "error: " << e.what() << std::endl;
      return false;
  }
}

int main()
{
  std::unique_ptr<questdb::ilp::line_sender> sender;
  try
  {
    sender = std::make_unique<questdb::ilp::line_sender>("localhost", 9009);
  }
  catch(std::runtime_error & e)
  {
      std::cerr << "error: " << e.what() << std::endl;
      return 1;
  }

  std::cout << sendRandomBuffer(sender.get()) << std::endl; // everything is awesome

  //wait to stop database (./questdb.sh stop)
  std::cin.get();

  std::cout << sendRandomBuffer(sender.get()) << std::endl; // success? HOW? Not the biggest problem anyway
  std::cout << sendRandomBuffer(sender.get()) << std::endl; // crash, the biggest problem

  return 0;
}

Flushing the buffer to the disconnected sender should throw an exception every time. Or there should be something like bool line_sender::is_connected() to check if I can flush the buffer to the sender.

[amunra]

Hi @malekva,

Disconnection is async due to protocol design, which is why you don't see the first sendRandomBuffer fail. They crash you're noticing later is likely an uncaught exception (can you provide a backtrace for it?, can you try surrounding the whole block with try / catch (const questdb::ilp::line_sender_error& qdb_err)?).

ILP is (for now) a one-way protocol: Messages are not ack'ed by the server so there is no way of knowing for the client to know that the TCP connection is dropped until it is actually dropped by the Linux/MacOS/Linux socket library. This may be a small amount of time after the server itself goes offline or closes the connection for other reasons (e.g. a data error). When a disconnect is eventually detected, this is reported as an exception in C++.

The folloing test simulates a disconnect and is pretty close to your example code: https://github.com/questdb/c-questdb-client/blob/main/cpp_test/test_line_sender.cpp#L414

See https://github.com/questdb/c-questdb-client#insertion-protocols-overview and https://questdb.io/docs/reference/api/ilp/overview/.

I hope this answers your question. Acks for the protocol and improved error reporting is certainly something we're looking to improve.

[amunra]

I haven't heard back in a while, so I'm closing this issue. If you have more information regarding the issue please re-open it.