Closed mikoskinen closed 6 months ago
I'm starting to wonder if this is more of an issue with the NATS server (or CLI?) and the .NET client is correctly creating the mirror KV.
Command:
nats s view KV_stest-invoices
Seems to list things correctly.
Though getting the key from the mirror KV doesn't work:
This is an issue with mirroring or filter subjects. I'm not sure how to resolve this yet.
Just a quick addition. Instead of mirroring I tried to use stest as the source for stest-invoices. It is showing identical behaviour:
Here's the code:
using var conn = new ConnectionFactory().CreateConnection();
var kvm = conn.CreateKeyValueManagementContext();
var kvName1 = "stest";
var kvName2 = "stest-invoices";
var k1 = new KeyValueConfiguration.KeyValueConfigurationBuilder()
.WithName(kvName1)
.Build();
kvm.Create(k1);
var src = new Source.SourceBuilder()
.WithName(kvName1)
.WithFilterSubject("$KV.stest.invoice.*")
.Build();
var k2 = new KeyValueConfiguration.KeyValueConfigurationBuilder()
.WithName(kvName2)
.WithSources(src)
.Build();
kvm.Create(k2);
Here's the stream report:
And here's the info for the stest-invoices:
C:\>nats s info KV_stest-invoices
Information for Stream KV_stest-invoices created 2024-03-12 12:56:20
Replicas: 1
Storage: File
Options:
Retention: Limits
Acknowledgments: true
Discard Policy: New
Duplicate Window: 2m0s
Allows Msg Delete: false
Allows Purge: true
Allows Rollups: true
Limits:
Maximum Messages: unlimited
Maximum Per Subject: 1
Maximum Bytes: unlimited
Maximum Age: unlimited
Maximum Message Size: unlimited
Maximum Consumers: unlimited
Replication:
Sources: KV_stest
Source Information:
Stream Name: KV_stest
Subject Filter: $KV.stest.invoice.*
Lag: 0
Last Seen: 147ms
State:
Messages: 1
Bytes: 128 B
First Sequence: 1 @ 2024-03-12 12:56:30 UTC
Last Sequence: 1 @ 2024-03-12 12:56:30 UTC
Active Consumers: 0
Number of Subjects: 1
Sorry for spamming... As the last idea I tried using "subject transform".
Unfortunately the behavior is the same: bucket reports no keys even though they are in the stream.
It's subject filter issue. I used JetStream directly to read the messages.
In the stream for stest
StreamInfo{Created=3/12/2024 5:43:33 PMTimestamp=3/12/2024 5:50:05 PM, StreamConfiguration, Stream:KV_stest | Message Subject: $KV.stest.invoice.500 | Data: invoice
Stream:KV_stest | Message Subject: $KV.stest.inv.500 | Data: inv
In the stream for stest-invoices
DeletedCount=0, FirstTime=3/12/2024 5:43:33 PM, LastTime=3/12/2024 5:43:33 PM}, ClusterInfo=null, NATS.Client.JetStream.MirrorInfo, SourceInfos=}
Stream:KV_stest-invoices | Message Subject: $KV.stest.invoice.500 | Data: invoice
The subject for the message in stest-invoices does not match the subject for a key in that bucket, so any KV operation, either through the cli or the .NET client will never find the key. It needs to be transformed. I'm working on figuring how to do it.
Observed behavior
Here's the complete C# code I'm using to create two KV buckets, where the second one is the mirror of the first one with a subject filter defined:
Now when using the stest bucket through CLI things seem to work OK:
But the unexpected behavior is when trying to list the keys in the stest-invoices bucket:
No keys found in bucket
Expected behavior
Listing the keys in the mirror bucket should work OK.
Server and client version
nats-server: v2.10.11 nats cli: 0.1.3 "NATS.Client" Version="1.1.4"
Host environment
Windows 11
Steps to reproduce