search

nats-io / nats.docs

NATS.io Documentation on Gitbook
https://docs.nats.io
Apache License 2.0
112 stars 237 forks source link

[FEATURE] Nats Client Protocol idempotency id (or sid) needed for all client commands instead of only PUB/HPUB #498

Open 6562680 opened 2 years ago

6562680 commented 2 years ago

I write PHP library to wrap nats. I've just detected one thing that can't be resolved in that protocol (maybe i just dont know how to do that)

In short:

  1. WRITE two CONNECT messages
  2. READ all contents from socket
  3. What INFO depend to what message?

Explanations:

Sockets is duplex. I can read from there and write there.

After sending any message i have two possible ways: 1) I start to read immediate after send command 2) I will read messages later

If am in (1), i usually receive answer for my command making duplex SYNC. But if we talk about (for example) VERBOSE mode - any command should send the response. Also we should remember that INCOMING message could become to socket between the answers of certain command. So even if i read SYNC the duplex - i could have unexpected remote message.

It forces me to create function like processUntil('+OK'). This command gets many of messages, run associated handlers until it gets for VERBOSE mode "+OK" answer. But what if there are two commands sent? I should expect TWO +OK messages and also i could get few unexpected PING, -ERR or MSG/HMSG messages between.

If i ignore that situation (wont create processUntil) i could receive -ERR between answers, for example 'Stale Connection', and continue to handle messages after but i shouldn't handle that behavior (cus of not client but server problem)

[ Removed due by comment below ]

As a temporarily solution - i will wrap it into "sentCommands/expectedAnswers" stack to solve, but it should be controlled via idempotence i guess. Like your PUB/HPUB defines sid to make subscribtion unique.

Also would be great to provide CONNECT argument "separator" to overwrite "\r\n" cus of body of message can have same separators and it would be easier to control separator letters. I solve it exploding incoming strings by "\r\n" and each step i try to resolve one of known commands. But actually it could be simplier.

Thanks.

6562680 commented 2 years ago

Also maybe it is PHP thing, not NATS, but i tried to wait until server send me -ERR 'Stale connection' and then write to socket something. Then i start to read messages. Unexpectedly i've received all pending responses from the socket and, in general, server response me. Why? If i received 'Stale Connection' according by docs - server should CLOSE the socket and maybe clear unread buffer. Even after that i receive all pending messages, error inside them, and can continue writing to socket. In advance socket was closed once i tried to READ from socket that send me "Stale Connection". So socket closes once i read, but not if i try to WRITE (i even checked socket remotely closed state and get FALSE to be sure). I repeat, maybe it is a PHP thing, but maybe a NATS one (maybe it is required to support JetStreams). Also i tried to write some message to 'Stale Connection' socket and didn't get the answer, seems correct. But if i send CONNECT first, socket becomes alive and getting commands again. What does it mean? I think we can create tonns of sockets, all of them could become 'Stale Connection', but (internally i dont know) still has in NATS registry with possible overflow or trigger "limitConnections" in future. Guess NATS just replace dummy sockets once alive socket gets in, but then we again could to receive unexpected situation where sockets was replaced and one of old replaced socket unexpectedly becomes alive and start to writing.

(above quote is not actual, sorry me, server didn't sent all pending messages, only new ones after receiving connect, possible PHP restores connection when i WRITE something to socket, guess is not nats problem)

6562680 commented 2 years ago

Guess i dont count parallel features. To escape responses from different request it should be different opened connections... So if we open new connection for each sid we work only with our messages, and messageId is not needed. That way we can escape blocking until one handler processing reply for his request and other handlers waiting for.

Guess each SUB message should open new connection, allowing engineer to create connection pool, that we check in a cycle with timeout. Same thing should be for PUB message with reply-to provided... There could be PUB with reply-to and then SUB with reply-to subject. User possible should add handler only for SUB openning new connection for each SUB then he receive answers only for his commands instead of all messages in one flow...

Actually noob mode and messageId would be great feature until user wants to ignore Pool feature, but currently it is very recommended.

Also adding handlers for just 'MSG' forces to run N handlers for each incoming message. Instead it should be some filter for message fields to run only 2-3 callbacks required for certain task. Well, if handlers started only by sid then we always know how many handlers we start.

One thing i hate in pool - highload. So many connections from single machine even with "parallel" could unexpectedly exhaust max_connection_limit or something. And what would we say if we start to talk about cluster... Connection count exponentially increased.

It will be harder than i expect.

6562680 commented 2 years ago

Tried async and sync, and just unable to solve case.

If confirmation/response doesn't have pointer to what exactly it covers - unable to work with these confirmations.

Ping should be confirmed with pong. We use socket, so when reading we could receive few pongs (in case of long time wait) - we again dont know what pong to what ping to be sure that it works.

Jarema commented 2 years ago

fyi there is already an actively maintained NATS PHP client: https://github.com/basis-company/nats.php with JetStream, KV etc.

6562680 commented 2 years ago

Thats why i rewrite it. There's not junior code, but not senior code, OOP cries, looping too, unable to async processing, am trying to explore.

Fully synchronyous blocking client with wall of code, but actually - few features is working. I tried to cooperate with - they mark task as "somewhen" and leave.