go-nitro Productization

[patrickwoodhead]

Open Grant Proposal: go-nitro Productization

go-nitro Productization

Proposal Category: core-dev

Proposer: George Knee

(Optional) Technical Sponsor: @patrickwoodhead

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT, APACHE2, or GPL licenses?: Yes

Project Description

This proposal builds on the work completed during the first go-nitro grant. The aim of this follow-on grant is to deliver a robust, efficient and production-ready minimal-viable state channel network client in Go; and proving its capabilities by deploying and benchmarking a testnet. The client will be connected to an adjudicator smart contract deployed on chain and tuned for high performance. The testnet will demonstrate an end-to-end interaction between retrieval clients and retrieval providers; locking funds on chain with a hub, spinning up virtual channels, executing micropayments, closing the virtual channel and eventually withdrawing funds on chain.

The grant is proposed to START on 5 July 2022. The deliverable dates are based on an estimate of the scale of the work (in developer days), the number of developers assigned to that milestone, and are shifted by a “time-off allowance” of 4 weeks.

Value

The importance of using multi-hop payments in the Filecoin retrieval market is clearly laid out in the retrieval market roadmap. (See for instance the Payment Channel Manager component in the V1 & V2 milestones.) Multihop channels leverage existing channels between peers in a payment network, and do not require new connections to be setup on chain. Virtual channel constructions offer a key advantage over the HTLC multihop construct present in Filecoin payment channels; once Alice & Bob construct a virtual channel through an intermediary Ingrid, Alice can pay Bob without involvement from Ingrid. Therefore the value of this grant will be to reduce latency for retrieval clients and retrieval providers, and to add much greater scalability to payment intermediaries and by extension to an incentivised retrieval network. We aim to showcase this value through data gathered in a realistic test environment.

Deliverables

1. Building and integrating the go-nitro payments API
2. Deploying an adjudicator on chain
3. Building a production chain service
4. Deploying and benchmarking a go-nitro test network
5. Achieving a “Time To First Payment” of less than 1s.

Development Roadmap

1. Building and integrating the go-nitro payments API

go-nitro does not currently allow any payments to actually be sent.

Deliverables

1. We will implement an API for creating/sending payments and an API for receiving/validating payments, implemented in golang.
2. We will provide an implementation of the supporting smart contracts.

We will aim to complete this milestone by MILESTONE 4 + (8 developer weeks / 1 developer) + (time-off allowance) = 22nd November 2022.

This milestone will be owned by Alex.

In implementing (1), we will prioritise shipping an early MVP for payments, with the understanding that it will probably change as we learn more about what the requirements (throughput, latency, storage constraints, …) are from integration partners.

Targeting an MVP / simplicity, we may:

• have unidirectional payment channels (If needed, the recipient can issue a “rebate” when the channel is closing)
• not include any “state” in the payments (e.g. file metadata)
• assume each deal uses a unique virtual channel

The minimum golang API would look something like:

type PaymentManager interface {
  // "I want to pay 5 more, relative to what I've paid so far"
  Pay(amount uint, channel ChannelId) (Payment, error)

  // OR: "I want to make sure Bob has been paid 55 in total"
  Pay(targetTotal uint, channel ChannelId) (Payment, error)
  ChannelBalance(channelId) Balance
}

type ReceiptManager interface {
  ReceivePayment(payment Payment) (paid uint, err error)
  ChannelBalance(channelId) Balance
}

The supporting smart contract will implement the IForceMoveApp interface, implementing a latestSupportedState function that will support the secure opening and closing of virtual channels, as performed by the virtualfund and virtualdefund objectives.

1. Unknowns
2. Will the client/provider send/receive the payments across the wire? Or will the PaymentManager? If the PaymentManager, do the client/provider need to relate individual payments with “units of service provided”?
3. Is a PaymentManager responsible for managing the payment channels required to facilitate payments? (eg. should channel be an argument to Pay?)
   • If so, how does it decide when to create a new channel? When to close an existing channel?
4. Will the go-nitro client implement PaymentManager and ReceiptManager, or will it be a separate package?

2a. Deploy an adjudicator to an EVM chain

We have already built a relatively mature codebase in solidity to govern the on chain side of nitro protocol. Given the maturity of that codebase, the maturity of Ethereum tooling (e.g. hardhat), and our experience with the ecosystem — modifying the existing solidity code offers a quick route to integrating on and off chain components of nitro. This is desirable to drive us toward realistic testing. There is also the possibility to reuse the solidity code on the FVM when support for that arrives (although we will not rely on that).

When this milestone is complete, the on-chain component of the improved nitro protocol — the actors required to support streamlined virtual channels — will be built, tested, and deployed to an EVM chain such as the Ropsten Ethereum testnet. We will use the native token of that chain to stand in for FIL.

We will aim to complete this milestone by START + (8 developer weeks / 2 developers) + (time-off allowance) = 30th August 2022.

This milestone will be owned by Colin / George.

2b. Deploy an adjudicator to an FVM chain

go-nitro does not currently allow FIL to be locked into a channel.

When this milestone is complete, the on-chain component of the improved nitro protocol — the actors required to support streamlined virtual channels — will be built, tested, and deployed to the nascent Filecoin Virtual Machine FVM.

We will aim to complete this milestone by START + (16 developer weeks / 2 developers) + (time off allowance) = 27th September 2022.

This milestone will be owned by George / Colin.

We will require ?$.

To deploy to an FVM chain, we will consider the following options:

1. Write an adjudicator in TinyGo, targeting WASM
2. Write an adjudicator in Rust, targeting WASM
3. Reuse our solidity code, targeting the EVM compatibility of the FVM

Option 2 carries with it the learning curve of Rust (we have very little experience with it). Option 3 includes the external dependency of solidity support (it is subject to FVM roadmap slippage).

3. Build a production chain service

When this milestone is complete, the off-chain and on-chain components of our system will work together in tandem. That is to say, “real” funds will locked on chain (before payments are sent off chain) and released again (after the retrieval deal completes).

We will aim to complete this milestone by MILESTONE-2 + (8 developer weeks / 1 developer) + (time-off allowance) = 29th November 2022.

This milestone will be owned by Mike.

We will require ?$.

To interface with an EVM deployment, we will use go bindings for our solidity contracts (using geth’s abigen utility).

To interface with an FVM deployment, we will need to research and become comfortable with the infrastructure and tooling around deploying to Filecoin. We may even be able to contribute to that infrastructure through our participation in the FVM early builders program.

The production chain service(s) will support both EVM and FVM on chain implementations.

Open questions:

1. Nitro states are encoded/decoded during the offchain as well as onchain code. The endoded states are signed in offchain code and signatures are validated in offchain and onchain code. How will encoding and signatures have to change to accomodate FVM?

4. Deploying and benchmarking a go-nitro test network

go-nitro is currently only battle-tested with a few instances running in a single process on a single machine.

When this milestone is complete, we will have a system which can spawn a large number of go-nitro clients in separate processes (e.g. on separate machines), have them interact with one another and record various benchmarks (such as TTFP, see Milestone 5) to a dashboard which can be shared with stakeholders.

We will aim to complete this milestone by START + (4 developer weeks / 1 developer) + (time-off allowance) = 30th August 2022.

This milestone will be owned by Alex.

We will require ?$.

This depends on us building a production messaging service to replace our current test-only messaging service. We have started work on this already, and have a messaging service that uses TCP servers.

We will write a testplan with testground, which is ideally suited to our needs here.

An extension will be to move to a messaging system which makes it easier to integrate with one or more retrieval market softwares (e.g. Lotus). This might be libp2p.

5. Drive down “Time To First Payment“

This depends on Milestone 4.

For the user, Time To First Payment (TTFP) is related to TTFB (Time To First Byte), because payments are the mechanism to incentivize retrieval provision. For we developers, TTFP corresponds to the time taken to complete virtualfund protocol. We need to keep TTFP small in order to ensure a good experience for retrieval clients.

In a network of 2 clients and 1 hub with no communication latency, we estimate the typical TTFP for go-nitro to be around 100ms . The behaviour for more realistic situations has not yet been measured. It is also not known how this will be affected by the completion of the other milestones.

When this milestone is complete, we will have an accurate idea of typical TTFP across multiple scenarios, and will have made improvements in order to drive it down. We hope to maintain sub 1s TTFP even with realistic network latency and many nodes operating.

We will aim to complete this milestone by MILESTONE-4 + (12 developer weeks / 1 developer) + (time-off allowance) = 29th November 2022.

This milestone will be owned by George.

We know of the following opportunities for optimization:

• Increase the amount of concurrency by judiciously inserting go keywords to allow the main engine thread to progress objectives in multiple Go routines. This will require concurrency control to prevent concurrent reads/writes to a given channel. We will need to perform refactors to keep everything working. This should allow for better vertical scaling, especially in hubs (which may can have several cores).
• Analysing the concurrency behaviour by using go-trace.
• Auditing the use of cryptography — for example removing unnecessary ECDSA recovery calls, unnecessary hashes, etc.
• Redesigning our ledger synchronization protocol, which is the expected bottleneck of a high throughput network.
• Auditing the use of buffered go chans versus other structures such as explicit queues.

We will measure the impact on throughput / latency by deploying to our test net.

Total Budget Requested

Managed offline

Maintenance and Upgrade Plans

Team

Team Members

George Knee

Colin Kennedy

Alex Gap

Mike Kerzhner

Team Website

Magmo Website

Relevant Experience

Collectively, Magmo has more than 15 years of experience thinking about state channels! As mentioned above, we successfully shipped a virtual funding protocol implementation in Typescript (powering Web3Torrent) and have already successfully built out most of our Golang implementation in a previous Filecoin Grant.

Our team has a healthy mix of academic backgrounds (PhDs in probability theory and quantum computing, one Masters in Computer Science) as well as software backgrounds.

Team code repositories

https://github.com/statechannels

[patrickwoodhead]

@realChainLife - I have posted this grant proposal here following our call with the Magmo team the other day.

[ErinOCon]

This is a completed grant project. The item has been closed.