[Question/Advice] Is persistance irrelevant for Greenlight (usage of `MemoryPersister`)

[lamafab]

Hello

I'm currently going through the source code and saw that Signer::new simply uses a MemoryPersister when constructing the vls_protocol_signer::handler::RootHandler.

        let persister = Arc::new(crate::persist::MemoryPersister::new());

        // ...

        let validator_factory = Arc::new(SimpleValidatorFactory::new_with_policy(policy));
        let starting_time_factory = ClockStartingTimeFactory::new();
        let clock = Arc::new(StandardClock());

        let services = NodeServices {
            validator_factory,
            starting_time_factory,
            persister: persister.clone(),
            clock,
        };

        let handler = handler::RootHandlerBuilder::new(network, 0 as u64, services.clone(), sec)
            .build()
            .map_err(|e| anyhow!("building root_handler: {:?}", e))?;

So is my assumption correct that persistence is not really relevant for Greenlight as long as you have the GreenlightCredentials?

I'm asking because I'm working on a project where we use the Greenlight API (as part of the Breez SDK), however, due to technical reasons we cannot use Rust to make networking calls directly. Meaning; Swift/Kotlin makes the networking calls and then passes on the data to Rust for processing. Our Rust code is basically in airgap and stateless mode (any state must be passed on by the caller, ie. Swift/Kotlin), and was wondering whether I can just use VLS directly instead of relying on this crate(?).

Thank you for any advice

[kingonly]

Why can't you use a different library with Kotlin/Swift bindings for these calls and then pass the data to the rust code? I mean, why does it matter that the underlying native library was written in rust?

[lamafab]

Why can't you use a different library with Kotlin/Swift bindings for these calls and then pass the data to the rust code?

Would you mind telling me which ones? If there are other Greenlight clients available then please let me know, in your docs I only saw Rust and Python.

[cdecker]

Our goal for the GL signer was that the seed alone should be sufficient to recover whenever possible. As such local storage on the signer cannot be relied on, but we may add it later to reduce the need for state syncing. In addition we support multiple signers that may be online at different times, making direct communication prohibitive.

This informed our choice to have a signer sync protocol: the signer maintains its state locally in the MemoryPersister, and not persisted to local storage, the SSS (signer state sync) protocol sends the state along with the signer request, the signer merges its in-memory persister with the differences, processes the request, and sends back a diff of prior state and post state back to the state server. The state will eventually be signed to protect against manipulation, the signer tracks a set of timestamps to protect against replay attacks, and ultimately we want the state server to be operated by someone else (the VLS team?) to further reduce the chances of colluding against the user (same rationale why Blockstream will likely not operate an LSP, since that prevents an entire class of collusions).

As for the need of Rust in your product, does that refer to the signer or the client? The signer is a self-contained software that can run inside or outside the main application. The client on the other hand uses the TLS identity to sign the payload, and add some metadata, that could be reimplemented in native clients, but having a core Rust logic that is then exposed via language-specific bindings means that we can implement it once and reuse it from everywhere, while maintaining multiple implementations of the clients. Of course that may be worth it if that unlocks enough use-cases, but we probably want to look how we can best use the existing parts we already have.

[JssDWt]

due to technical reasons we cannot use Rust to make networking calls directly. Meaning; Swift/Kotlin makes the networking calls and then passes on the data to Rust for processing. Our Rust code is basically in airgap and stateless mode (any state must be passed on by the caller, ie. Swift/Kotlin)

@lamafab Could you elaborate on the technical reasons behind this? I'm asking because if that's a problem for you, it might be a problem for others as well.

[kingonly]

Why can't you use a different library with Kotlin/Swift bindings for these calls and then pass the data to the rust code?

Would you mind telling me which ones? If there are other Greenlight clients available then please let me know, in you docs I only saw Rust and Python.

I mean you can use the Breez SDK Kotlin/Swift bindings

[cdecker]

The existing languages for bindings are:

• Rust (GL, Breez SDK)
• Python (GL, Breez SDK)
• Javascript (there are bindings in GL that went unmaintained that but we can resurrect them or rebuild them)
• C# (Breez SDK)
• Kotlin (Breez SDK)
• Swift (Breez SDK)
• Android (Breez SDK)

[lamafab]

@JssDWt: @lamafab Could you elaborate on the technical reasons behind this? I'm asking because if that's a problem for you, it might be a problem for others as well.

This situation is generally specific to us. The project we work on, "Wallet Core", is a low-level library (mostly C++, but we're slowly migrating to Rust) that constructs all the transactions for various networks. It's basically an isolated blob with no networking or storage. For example in case of Bitcoin, a higher level application (such as an Android or iOS app) fetches the UTXOs from the network, then passes that information on to Wallet Core which builds, constructs and serializes the final transaction, which is then returned to the caller who then submits it to the network.

However, based on what I'm seeing here it might be a better/easier decision to skip the Wallet Core part and just use the BreezSDK + Greenlight in the higher-level application directly via the bindings, which is generally unconventional for us. But I need to evaluate this further and talk to some people on whether this is something we can/should do.

@cdecker thanks for the explanation, looks like the state syncs do add quite a bit of complexity. Depending on how we proceed I might have to look into this further.

Anyway, thanks for the quick responses!

EDIT: And when it comes to Greenlight, my initial idea for doing the authentication/signing/etc in Rust is because I don't want to put this burden on the Android/iOS devs. Respectively, from their perspective the network messages are just opaque blobs that they pass on to Wallet Core, which constructs the credentials and processes all network messages.

[cdecker]

I think we are overall aligned in how we think about things (taking on the difficult parts so that devs using our library don't have to jump through many hoops).

I'd like to highlight that it is very much possible to separate the transport from the actual processing of requests, however complex these may be. We currently use mTLS to authenticate the client directly to the node via a direct connection, but we're working on reducing that requirement, and replace the authenticated connection with authenticated payloads. This allows us to interrupt the actual connection, and have other components on the path between signer and node, or client and node. If you are wondering, yes, that was also the reason why the authenticated payload was introduced in the first place: it is often not possible to establish a direct connection between signer and client, therefore we need to communicate through the node which should be reachable from anywhere, but now we can't use an authenticated connection because we aren't connecting directly anymore, hence why we introduced authenticated payloads to prove to the signer, that the payload originated from an authorized client, and was not injected by the node.

Long story, short: we can tunnel the actual message transport however we want, so your project could take the gl-client library, split it into transport and processing, and then replace the transport with your own transport used to communicate with the rest of the network. Payloads would just be opaque to you, so we don't start depending on each other, and all you need to do is call a rust function on gl-client with the opaque request and return the opaque response to the node. You are in control of connections, and the gl-client processing part does not need to know anything about comms.

This is also a way we are planning to enable embedded devices to talk to GL by the way, since they mostly don't have networking stacks we end up having a companion app somewhere in charge of communicating with the GL node, while the embedded device receives the request from the companion, e.g., over serial port, and returns the response to the companion which delivers it at the node.

Does that sound like an option?

(closing this issue soon, if no additional feedback is required, since it is not addressable via code)

[lamafab]

Hey @cdecker, thanks for your reply.

Long story, short: we can tunnel the actual message transport however we want, so your project could take the gl-client library, split it into transport and processing, and then replace the transport with your own transport used to communicate with the rest of the network.

Yes, this was initially the idea, given that this is how we usually do things. However, in our project(s) we now also rely a lot on Kotlin/KMP, so using the BreezSDK bindings ended up working great for us. We reached all targets for this quarter. Therefore we don't need anything to be done and this issue can remain closed. Nice to hear that you're working on splitting it anyway for embedded devices, though.