nats-io / nats.net

Full Async C# / .NET client for NATS
https://nats-io.github.io/nats.net
Apache License 2.0
233 stars 48 forks source link

NATS.net JetStream singleton issues #620

Open Bykiev opened 2 weeks ago

Bykiev commented 2 weeks ago

Observed behavior

When creating a singleton connection to NATS JetStream the error is thrown:

NATS.Client.JetStream.NatsJSApiNoResponseException: No API response received from the server
at NATS.Client.JetStream.NatsJSContext.JSRequestAsync[TRequest,TResponse](String subject, TRequest request, CancellationToken cancellationToken)
at NATS.Client.JetStream.NatsJSContext.JSRequestAsync[TRequest,TResponse](String subject, TRequest request, CancellationToken cancellationToken)
at NATS.Client.JetStream.NatsJSContext.JSRequestResponseAsync[TRequest,TResponse](String subject, TRequest request, CancellationToken cancellationToken)
at NATS.Client.JetStream.NatsJSContext.GetStreamAsync(String stream, StreamInfoRequest request, CancellationToken cancellationToken)
at NATS.Client.KeyValueStore.NatsKVContext.GetStoreAsync(String bucket, CancellationToken cancellationToken)

My config:

builder.Services.AddNats(1, opts =>
{
    return new NATS.Client.Core.NatsOpts()
    {
        Name = name,
        Url = url,
        SerializerRegistry = NatsJsonSerializerRegistry.Default
    };
});

We've tried to use a connection pool with 1000 connection and everything is working as expected. Maybe it's not possible to subscribe and update the same store using one connection? I didn't find any information about it in the docs. And if it's not possible, how many connections should be in a connection pool?

Expected behavior

No exception is thrown when NATS is registered as singleton

Server and client version

NATS server 2.10.18 NATS .NET v2.3.3

Host environment

Docker image

Steps to reproduce

No response

mtmk commented 2 weeks ago

we see this error sometimes. I'm assuming it happens intermittently and you have other consumers on the same connection. it may be head of line blocking. my guess is you can use keyed services and use a seperate connection for your consumers which may flood the connection.

edit: we also had reports of network issues e.g. using public IP of your cloud VM which might have DDoS protection

Bykiev commented 2 weeks ago

Yes, we have a multiple consumers. We did some tests and I can see this error even with 10 connections in a pool. We've updated the NATS server docker image to v2.10.19, seems the problem still exists. What can we do to use NATS JS with multiple consumers?

Bykiev commented 2 weeks ago

After some time we didn't get any data via JS on some of the consumers, it seems the connection is dropped by someone or smth else is happened.

Bykiev commented 2 weeks ago

I did some research and this seems to be DI or pooler issue. I've removed the DI part from the project and created a single connection for watcher and another one for writing. This is working without any issues.

mtmk commented 2 weeks ago

I did some research and this seems to be DI or pooler issue. I've removed the DI part from the project and created a single connection for watcher and another one for writing. This is working without any issues.

this is interesting. pooler is just a simple round robin of connections. are you on slack.nats.io maybe we can have a chat there.

edit: you should also be able to used keyed DI in you're using net8

Bykiev commented 2 weeks ago

are you on slack.nats.io maybe we can have a chat there

Unfortunately, It's not available in my country. I've created another issue https://github.com/nats-io/nats.net/issues/621, can be related to this issue