Republish does not work for a stream with no consumers and Interest retention policy

[vazhem]

Observed behavior

Configuration: streamA(source) -> streamB ---republish---> streamC1,C2,C3

Configured supercluster with jetstreams. streamA is a source for streamB. streamB has no consumers but republishes messages to streams streamC1,C2,C3.

If set retention policy for streamB to Interest then messages written to streamA are received by streamB but dropped and not republished to streams streamC1,C2,C3. (With Limits retention policy on streamB everything works as expected).

Expected behavior

If a stream sources messages from another stream, has no consumers, has retention policy Interest, republishes messages to other streams - republishing should work and messages should not be dropped before they are republished.

Server and client version

embedded nats server v2.10.9

Host environment

No response

Steps to reproduce

No response

[neilalexander]

Thanks for the report! I've had a look into this further and we've spoken internally and believe that the behaviour is correct.

Message republishing would be done after a sequence has been assigned to the message and a timestamp set, in part because those are populated into the headers of the republished messages. This would only be done if the message was actually stored, so for a stream with an interest retention policy, there would need to be interested consumers for that to happen.

If you don't need to consume directly from stream B then using limits-based retention with a max age, max msgs etc should be fine just to prevent it from growing endlessly.

[vazhem]

Using Limits retention is a workaround which seems to work. Will it work with limit of 1 message and remove old policy? If not - what should be the minimum size of the stream to ensure that messages are republished? Or there is no strong guarantees for republishing a message to another stream anyway? (For example message might not get republished if a node crashed after adding message to internal async memory queue for republishing, but did not get a chance yet to republish it)

[ripienaar]

Republish uses nats core publish and is at most once. So it’s unreliable.

For stream to stream better use source or mirror

[vazhem]

Thanks for the information, it is helpful. You may close this defect if everything works as expected.

[derekcollison]

@ripienaar is correct that messages can be dropped, however the headers in each msg do contain Nats-Last-Sequence which is the last sequence you should have seen, so you can detect gaps. This is not taken into account for a stream processing republishes from another stream though at this time.

[vazhem]

Thanks for additional info about how to detect gaps. It would be useful if we would wish to implement stronger guarantees on top of unreliable republishing, but that would require to store messages in a separate store and have periodic acknowledgements from target that it has received all messages or we need to resend some... For now as it was mentioned republishing feature has only 'At most once' delivery guarantees, which should be fine for some use cases.