search

nats-io / nats.c

A C client for NATS
Apache License 2.0
382 stars 132 forks source link

[DO NOT MERGE] single threaded nats client POC #769

Open levb opened 1 month ago

levb commented 1 month ago

PR for discussion only, not to be merged.

Example: https://github.com/nats-io/nats.c/blob/mininats/examples/libevent-pub.c It connects and sends 5 messages to a server. The entire heap usage is logged (there are still leaks, please ignore, cleanup is not fully implemented).

To see more details, turn on more logging here

➜  build git:(mininats) ./bin/nats-libevent-pub
TRACE:   HEAP: allocated bytes:4096, ptr:0x153008800 (nats_log_createPool:mem_pool.c:600)
Hello, mini-nats! Will try to send a message to 'foo'
TRACE:   HEAP: allocated bytes:4096, ptr:0x15300b800 (nats_log_createPool:mem_pool.c:600)
TRACE:   HEAP: allocated bytes:4096, ptr:0x15300d800 (nats_log_createPool:mem_pool.c:600)
TRACE:   HEAP: allocated bytes:4096, ptr:0x154008200 (nats_log_createPool:mem_pool.c:600)
TRACE:   HEAP: allocated bytes:65536, ptr:0x158008000 (natsPool_getReadBuffer:mem_pool.c:216)
TRACE:   HEAP: allocated bytes:4096, ptr:0x154009200 (_allocSmall:mem_pool.c:135)
TRACE:   HEAP: allocated bytes:4096, ptr:0x154808200 (nats_log_createPool:mem_pool.c:600)
Message enqueued, text: Hello, NATS!
TRACE:   HEAP: allocated bytes:4096, ptr:0x154809200 (nats_log_createPool:mem_pool.c:600)
Message enqueued as a copy, freeing the data now: Hello, NATS! I was allocated on the heap, my bytes were copied into another buffer, and I was freed immediately by the app.
TRACE:   HEAP: allocated bytes:4096, ptr:0x15480a200 (nats_log_createPool:mem_pool.c:600)
Message enqueued, text: Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was freed by a cusom 'message published' callback
TRACE:   HEAP: allocated bytes:4096, ptr:0x15480b200 (nats_log_createPool:mem_pool.c:600)
Message enqueued, text: Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was 'free'-ed by the cleanup, no custom callback needed.
TRACE:   HEAP: allocated bytes:4096, ptr:0x15480c200 (nats_log_createPool:mem_pool.c:600)
FINAL message enqueued, text: Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was 'free'-ed by the cleanup, no custom callback needed.
TRACE:   HEAP: freeing ptr:0x154808200 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x154809200 (_free:mem_pool.c:84)
Message published, freeing the data: Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was freed by a cusom 'message published' callback
TRACE:   HEAP: freeing ptr:0x15480a200 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x15480b200 (_free:mem_pool.c:84)
Last message published, now disconnecting:
Connection closed, last error: ''
TRACE:   HEAP: freeing ptr:0x154008200 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x154009200 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x15300d800 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x15300b800 (_free:mem_pool.c:84)
TRACE:   HEAP: freeing ptr:0x153008800 (_free:mem_pool.c:84)

server log:

[70659] 2024/06/24 08:52:14.781129 [TRC] [::1]:59103 - cid:85 - <<- [CONNECT {"verbose":false,"pedantic":false,"tls_required":false,"name":"","lang":"C","version":"4.0.0-mininats","protocol":1,"echo":true,"headers":true,"no_responders":true}]
[70659] 2024/06/24 08:52:14.781167 [TRC] [::1]:59103 - cid:85 - <<- [PING]
[70659] 2024/06/24 08:52:14.781169 [TRC] [::1]:59103 - cid:85 - ->> [PONG]
[70659] 2024/06/24 08:52:14.781292 [TRC] [::1]:59103 - cid:85 - <<- [PUB foo 12]
[70659] 2024/06/24 08:52:14.781296 [TRC] [::1]:59103 - cid:85 - <<- MSG_PAYLOAD: ["Hello, NATS!"]
[70659] 2024/06/24 08:52:14.781301 [TRC] [::1]:59103 - cid:85 - <<- [PUB foo 123]
[70659] 2024/06/24 08:52:14.781351 [TRC] [::1]:59103 - cid:85 - <<- MSG_PAYLOAD: ["Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was freed by a cusom 'message published' callback"]
[70659] 2024/06/24 08:52:14.781368 [TRC] [::1]:59103 - cid:85 - <<- [PUB foo 123]
[70659] 2024/06/24 08:52:14.781382 [TRC] [::1]:59103 - cid:85 - <<- MSG_PAYLOAD: ["Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was freed by a cusom 'message published' callback"]
[70659] 2024/06/24 08:52:14.781387 [TRC] [::1]:59103 - cid:85 - <<- [PUB foo 130]
[70659] 2024/06/24 08:52:14.781391 [TRC] [::1]:59103 - cid:85 - <<- MSG_PAYLOAD: ["Hello, NATS! I was allocated on the heap, my bytes were NOT copied, and I was 'free'-ed by the cleanup, no custom callback needed."]
[70659] 2024/06/24 08:52:14.781398 [TRC] [::1]:59103 - cid:85 - <<- [PUB foo 0]
[70659] 2024/06/24 08:52:14.781402 [TRC] [::1]:59103 - cid:85 - <<- MSG_PAYLOAD: [""]
[70659] 2024/06/24 08:52:14.781409 [DBG] [::1]:59103 - cid:85 - Client connection closed: Read Error
levb commented 1 month ago

@kozlovic ACK that it is way too early to review, and I am focused on other things for the time being anyway. I marked you to get your "FYI"-level attention, feel free to un-assign if this is too noisy.

...each publish ended-up result in the tcp write

the plan was to optimize it to use writev, like in https://github.com/nginx/nginx/blob/e734df6664e70f118ca3140bcef6d4f1750fa8fa/src/os/unix/ngx_writev_chain.c#L14 but I didn't get to prototype it. So everything would still be buffered (into a chain), and then segments written out of the chain, 1 iovec at a time per write event. I just didn't get to the sockets at all, figured I'd wrap it on top of the existing code for now. Ditto establishing connections - that too would need to be event driven for re-connections.