Golang client

[darioAnongba]

Hi everyone and thank you for this amazing implementation enabling an easy self-custodial software development on the Lightning network.

I recently started playing with Greenlight as I got my certificate by registering with the Developer Console. I am trying to implement Greenlight in a Golang application, so I can't use the Rust or Python clients. I encountered different problems while trying. This is my experience.

Breez SDK

First, I considered the Breez SDK (breez-sdk-go). I thought it was mainly a client to the Greenlight service. I just wanted to retrieve data from my Greenlight node and create invoices/send payments. To my surprise, to use the Breez SDK one needs to get an API Key by contacting Breez.

Moreover, the Breez Go SDK doesn't seem to contain the "Register" function.

No Go client

Then I understood that the Breez SDK is more than just a Greenlight client and has many functionalities that I don't need, given that I already have a Greenlight certificate, I should be able to use Greenlight directly without using the Breez SDK.

Sadly, there is no Go client provided in the Greenlight repo. This shouldn't be a major issue since you provide .proto files that I should be able to use to communicate directly with your service.

GRPC

I then decided to use the .proto files and generate .go files enabling me to communicate with Greenlight. This worked fine and I was able to get nice packages to use in my app (pb files) with protoc.

I started my implementation by looking at the Rust code and following the steps. Sadly I was blocked at the sign_message part after receiving a challenge. I couldn't implement my own signer as I'm missing info on signature scheme and encoding. Given that this has already been implemented in Rust and Python, it would be better to reuse that code.

Finally, I tried using the Breez SDK to sign the challenge but for some unknown reason you first need to call Connect before you can use SignMessage, which maybe is a design problem? (Shouldn't I be able to sign as long as I have a seed regardless of my ability to connect to Breez services?)

Ask: Go package wrapping Rust

So I gave up as it seems that I'm blocked with my Go implementation unless I can get a Go client that will wrap the core Rust one. I'm happy to contribute to this repo if you think that this is needed. Otherwise, is there any other solution? The way I see it, the signer functionality is the piece I need most.

Thank you for your time, Dario

[cdecker]

Hi @darioAnongba, we are looking into ways to create more bindings with less maintenance overhead (and with less chance of accidentally introducing a discrepancy between the languages). So far we've been looking into uniffi which seems to fit rather nicely with our goal of describing the API once and exposign that API in a variety of languages.

Golang does not appear to be one of the languages supported by core uniffi, however the templating system seems to suggest it's rather simple to generate these bindings from the API description. And there is at least one template for golang that appears like it could work: https://github.com/NordSecurity/uniffi-bindgen-go

@adi-shankara is currently looking into whether uniffi is the way we want to go, and whether we want a narrow or a wide interface.

• Narrow interface: expose just a call method that takes the grpc URI and encoded payload, and then return the encoded response. This means the FFI boundary is very narrow, and the encoding and decoding of the protobuf payloads needs to be done in the host language.
• Wide interface: expose each service and its methods in the API description, meaning we have quite a few methods that cross the FFI boundary, but we probably don't have to even touch the generated code, since uniffi takes care of payload transformation and call dispatching itself.

Do you happen to have experience with uniffi?

[darioAnongba]

Hi @cdecker, Thanks for the reply. Yes I know uniffi and I think that it's a good decision to use it to create different language bindings. From what I've seen recently in the Bitcoin developer ecosystem, many teams are going for a core Rust implementation and using uniffi to create bindings, some examples:

• Breez SDK is built in Rust (breez-sdk) and use uniffi to create breez-sdk-go, breez-sdk-swift, etc.
• RGB folks have built an sdk called rgb-lib and use uniffi to create bindings such as rgb-lib-kotlin, rgb-lib-python, etc.

So using uniffi seems a given, now the question is what should you use uniffi for. Of course I won't mingle in your design decisions but as a software architect like yourself, I will opt for the wrapping only the core functionalities, my reasons are:

• Separation of concerns: You should (IMO) not feel responsible for everything, as long as you provide the interfaces (the .proto), the core components such as the cryptograhic functions (like sign_message, sign_invoice, ...) and extensive documentation, then the community will quickly create their own packages around it.
• Different practices: Each language has different practices and nomenclature, so it is difficult to follow these practices the bigger the package.
• Maintenance overhead: As you mentioned, you want to avoid maintenance overhead, so the more you provide, the more you have to maintain when your focus should be improving the core Rust package.

So I wouldn't even implement the call part at all. Implementing a caller/RPC client from .proto files is trivial. This is code I produced in less than 10 minutes by using your .proto files and follows good practices in Golang, as you can see, what I need is a signer.Signer, the RPC calls are no problem:

package greenlight

import (
    "context"
    "log"

    "github.com/bitcoin-numeraire/wallet/src/adapters/lightning"
    "github.com/bitcoin-numeraire/wallet/src/adapters/signer"

    "github.com/bitcoin-numeraire/wallet/proto/scheduler"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials"
)

type Client struct {
    conn   *grpc.ClientConn
    client scheduler.SchedulerClient
}

var _ lightning.LightningClient = &Client{}

func New() (lightning.LightningClient, error) {
    // Create TLS credentials
    creds, err := credentials.NewServerTLSFromFile("client.crt", "client-key.pem")
    if err != nil {
        return nil, err
    }

    // Dial server
    conn, err := grpc.Dial("scheduler.gl.blckstrm.com:443", grpc.WithTransportCredentials(creds))
    if err != nil {
        return nil, err
    }

    client := scheduler.NewSchedulerClient(conn)

    return &Client{client: client}, nil
}

func (c *Client) Register(ctx context.Context, signer signer.Signer) error {
    challenge, err := c.client.GetChallenge(ctx, &scheduler.ChallengeRequest{
        Scope:  scheduler.ChallengeScope_REGISTER,
        NodeId: signer.NodeID(),
    })
    if err != nil {
        return err
    }

    log.Printf("Got a challenge: %v", challenge.GetChallenge())

    signature, err := signer.SignChallenge(challenge.GetChallenge()) // that's where I'm stuck because signer doesn't exist in Go
    if err != nil {
        return err
    }

    log.Printf("Challenge signed: %v", signature)

    resp, err := c.client.Register(ctx, &scheduler.RegistrationRequest{
        Network:   "bitcoin",
        Challenge: challenge.GetChallenge(),
        Signature: signature,
        NodeId:    signer.NodeID(),
        etc...
    })
    if err != nil {
        return err
    }

    // Use the response
    log.Printf("Response: %v", resp)

    return nil
}

func (c *Client) Close() error {
    err := c.conn.Close()
    if err != nil {
        return err
    }

    return nil
}

Here signer.Signer is the interface that I want to take from the Rust core client. The RPC calls in the code above are actually working fine. I tried to use the Breez SDK as a Signer but it doesn't work.

[cdecker]

Thanks @darioAnongba, I totally agree with all your points. We would love for the proto file to be the interface, however the Rust library does a bit more magic than a proto generator will generate for us: the E2E verification requires the protobuf payload for the grpc call to be signed with the client certificate.

This is implemented as a client-side middleware in as the AuthLayer and AuthService structs in service.rs. This will basically do four things:

1. Take the serialized payload and copy it into a buffer
2. Load the private key matching the user certificate from the in-memory copy
3. Sign the payload, and add the signature and a nonce to the HTTP2 headers
4. Forward the cached payload to the node

This is used on node to:

1. Copy the serialized payload before deserializing it and the associated signature and nonce
2. Store these into the signer context (the signer independently verifies the authenticity of the grpc call, and matches it up with the observed channel changes, rejecting any change it can't find a justification for)
3. Attach the signer context to any signer request that comes through the plugin on its way to the signers
4. Removes the call from the context once the call returns

If we want to generate the bindings out of the proto file we'll have to reimplement this middleware (and any that we may add in the future) for each language. If we use the Rust core, we need a way to pass the call to it, which is where the FFI comes in, and the distinction between narrow interface (only map call and then write adaptors in each language) and wide interface (generate native methods that individually are mapped through the FFI to the Rust core).

So as far as I can see the options we have are the following:

• Generate from proto + reimplement the middleware for each language (requires work in each language)
• Use uniffi to describe an FFI interface, and then we have one more choice:
  • Generate a narrow interface (only call) and then write adaptors (requires maintaining each language's adaptors)
  • Generate a wide interface (map each method) and then write a generator that generates the WebIDL file from the proto file (we have to maintain a generator, but at least that's for all languages, and we don't have to touch the bindings afterwards)

I think that last option is likely the best option. Also keep in mind that the signer also needs to be controlled from the host language, and for that we'd need an FFI interface anyway, because that one we definitely don't want to re-implement multiple times.

[darioAnongba]

Thanks for the complete answer. I understand better now. I seems that the last option is indeed a good one. I'll let you close this issue then and I'll wait for the bindings, I am using Rust at the moment so don't need Go anymore.