Add NATS replica client

[delaneyj]

NATS is a distributed application framework. It has the ability to store objects similar to S3/Minio in a replicated persistence store. This replica client supports this feature for Litestream.

Note The replica_client_test.go was geared towards only being tested against file. Have moved from fs/os errors to Litestream specific ones. This meant updating the file replica client to use these as well.

[betamos]

Big NATS fan here (oh, and Litestream too of course). Just stumbled across this PR accidentally and looked through it briefly. Some thoughts:

• Why auto-create the bucket? IIUC litestream wants you to provision resources yourself (like the S3 client), which is trivial with the NATS CLI. If litestream is resposible, NATS has tons of config options that could pollute the litestream config in the future (currently I see replica count, which isn't necessary for litestream to know about).
• Related: Retention is set on the NATS bucket. Without official support for out-of-bounds retention, I think it's much safer to let litestream manage retention in the same way it does elsewhere, using periodic checks.
• Only JWT authentication? I think user/pass should absolutely be supported as a base-case, it's very common. (It can technically be encoded in the URL but since user/pass fields exist in litestream.yml, it violates the principle of least surprise).

Apologies for stepping in unannounced. Let me know if I can help out any other way.

[delaneyj]

@betamos thanks for taking the time

• Why auto-create the bucket? IIUC litestream wants you to provision resources yourself (like the S3 client), which is trivial with the NATS CLI. If litestream is resposible, NATS has tons of config options that could pollute the litestream config in the future (currently I see replica count, which isn't necessary for litestream to know about).

I can see it both ways, if it's a blocker I'll remove it.

• Related: Retention is set on the NATS bucket. Without official support for out-of-bounds retention, I think it's much safer to let litestream manage retention in the same way it does elsewhere, using periodic checks.

This is a good point. In general you want to set limits on buckets but this goes to your previous point.

• Only JWT authentication? I think user/pass should absolutely be supported as a base-case, it's very common. (It can technically be encoded in the URL but since user/pass fields exist in litestream.yml, it violates the principle of least surprise).

I teach NATS usage at Synadia to customers. We highly recommend JWT only. User/pass, while supported currently support it really shouldn't be used. It's very easy to create JWT programmatically and have far superior control available.

Apologies for stepping in unannounced. Let me know if I can help out any other way.

Not at all, I think Litestream is an amazing tool and should be available in a NATS centric pipeline.

[stumct-sa]

I was looking through the litestream docs to see if it supported NATS then came across this PR. Would be great to see this get over the line.

I agree with the previous comment about the bucket configuration. I would expect to configure my own bucket and just tell litestream which one to use. There is a number of bucket configurations that will be set depending on my use case at the time. I suggest just taking in a bucket name and expecting it to exist.

JWT authentication is my preferred approach but again I wouldn't have litestream be so opinionated. It should support any NATS auth mechanism

[delaneyj]

I have been slammed with other projects but I agree with the comments above and will remove the bucket config from this sorry and look for a review