Call-01 Agenda - 2024-08-29

[hthillman]

Intro - 5-10min

Welcome & context-setting, plus Introductions

Shareouts - 20-30min

Please hold questions during this time.

AI SPE

Livepeer Cloud SPE

Livepeer Studio

Aquareum

Open Share

Open Discussion - 30min

• Improving AI Release Communication Due to custom forks of go-livepeer, the Livepeer.Cloud SPE needs time to deploy and upgrade the custom updates. Orchestrators need time to upgrade in order to keep up with Gateway Minimum Version Requirements

• SDK Release

  • Align release schedules

• PR Review capacity as blocker

• The current AI version of go-livepeer requires the team to investigate ongoing issues with the selection algorithm.

  • Models/Pipelines fail on the Livepeer.Cloud AI Gateways and our team has to investigate.
  • Requirements Planning for AI Metrics and Visibility has required a custom fork of go-livepeer to enable proper selection for AI Job testing

• Streaming interfaces (e.g. continuous text-to-audio)

• Signaling

  • Supply signaling for Gateways
  • Demand signaling for Orchestrators

Wrapup & Goal-Setting - 15min

[iameli]

Aquareum

Priorities

Outline your top 3 development priorities (1-3 sentences max)

I just finished up the tooling for C2PA signing using an Ethereum wallet; I can take a livestream, slice it up into one-second MP4 files, and sign each segment with an Ethereum identity. Now that we have that, the priorities are:

• Hook MistServer up to the stream of MP4 files so it can facilitate user playback.
• Prototype decentralized replication, potentially leveraging the B <> O gRPC wire protocol.
• Integrate Livepeer transcoding. Initially this will be a relatively shallow integration but the plan is to eventually upgrade the Livepeer Network to work directly with these primitives.

Blockers

Are there any external factors preventing you from achieving your top 3 priorities? If so, please outline them (1-3 sentences max)

• The C2PA spec doesn’t support secp256k signatures, so nothing signed with a Ethereum Wallet or a Livepeer Orchestrator can exist within their universe[^1]. Technically everything I’m doing goes against the spec. In practice, I’ve forked the primary Rust repo and CLI tool (c2patool) and it works flawlessly. I’ve been bugging people to try and get secp256k added to this list but they’re not really listening to me.

• The new World of Warcraft expansion came out yesterday ¯_(ツ)_/¯

Discussion topics

Got a topic you’d like to cover in open discussion?

We may not be able to cover every single topic that’s raised, but we will make sure that anything we miss is added as an agenda item for the next time.

[^1]: They have support for SHA-256 ECDSA, but only over the secp256r1 curve, not the secp256k1 curve. Distinction explained here. Supporting both curves in Rust is trivial; in Go it’s more of a PITA but I’m making it happen.

[mikezupper]

Livepeer.Cloud SPE

Priorities

Outline your top 3 development priorities (1-3 sentences max)

1. Livepeer.Cloud SPE Proposal (AI Metrics & Visibility)
   • Requirements Planning & Architecture Design - This is underway.
   • Coding / Implementation - The team is prototyping and will begin once all requirements and architecture is finalized
   • Testing and Validation - Not started
   • Production Deployment & DevOps - Not started
2. Livepeer.Cloud Infrastructure Migration to Kubernetes
   • In preparation for the Production Deployment above, the existing infrastructure will be migrated to Kubernetes. This work will to begin this week.
3. Livepeer.Cloud SPE - AI & Transcoding Gateway Support / Maintenance
   • Each day the Livepeer.Cloud Team looks into open issues reported by the AI SPE Team. This has been variable but seems to be increasing as usage increases on the network and with each release of go-livepeer.

Blockers

Are there any external factors preventing you from achieving your top 3 priorities? If so, please outline them (1-3 sentences max)

1. The current AI version of go-livepeer requires the team to investigate ongoing issues with the selection algorithm.
   • Models/Pipelines fail on the Livepeer.Cloud AI Gateways and our team has to investigate.
   • Requirements Planning for AI Metrics and Visibility has required a custom fork of go-livepeer to enable proper selection for AI Job testing

Discussion topics

Got a topic you’d like to cover in open discussion?

1. Improving AI Release Communication
   • Due to custom forks of go-livepeer, the Livepeer.Cloud SPE needs time to deploy and upgrade the custom updates.
   • Orchestrators need time to upgrade in order to keep up with Gateway Minimum Version Requirements

[victorges]

Livepeer Studio

Priorities

• AI-only SDK for any gateway, using Speakeasy similar to Studio
• Add go-livepeer AI APIs to the Studio SDK (it's a compatible provider)
• Follow AI network development closely and ensure high quality standards on go-livepeer code

Blockers

• Finalizing the OpenAPI schema handling on the AI network services
• (almost done) Reaching consensus on how the SDKs should be created/maintained

Discussion topics

AI selection algorithm

• It's really hard to get successful responses from the Studio gateway if we don't force the request to go to an EU region. We need to update the selection algorithm to account for the different requirements of the AI jobs compared to video transcoding. What needs to change?
• Related to the above, how do we optimize that algorithm to also minimize the model warm-up problem assuming a fully decentralized network?

Streaming workflow

• How much of a lift would be to create some streaming interfaces for models that support it? Think of continuous audio-to-text or even image-to-image manipulation (e.g. from a livestream)

[rickstaa]

AI SPE

Priorities

• Merge AI Remote Worker
  First review of go-livepeer#3088 completed by @rickstaa and Yodon. Final reviews will be done by @rickstaa and @thomshutt. @ad-astra-video is writing up a report to state the comparison with [Nico's impelmentation](https://github.com/livepeer/go-livepeer/pull/3106. @rickstaa will set up a meeting to start next week. Deadline: 6th Sept.

• Release Segment Anything 2
  Requested by a high-demand startup. The full pipeline is implemented and set for release on Friday. @rickstaa's review is still pending. Deadline: 1st Sept.

• Release Lip Sync and Text-to-Audio Pipelines
  Almost complete but facing dependency issues. The initial endpoint needs to be up by 1st Sept, with the full release on 6th Sept.

• Merge Selection Algorithm Fixes
  Found in PR 3033.

• Investigate Stopped Container Selection Issue
  Address the problem detailed in AI-588.

• Community SDK Release
  Needed for startup integration (Python and TypeScript). Deadline: 6th Sept.

• Setup Ecosystem Infrastructure

Blockers

• Developers
  The above tasks require my team to work hard, but the deadlines are still doable. However, we also have additional priorities to address, including streamed responses, improved networking and selection, better error communication, model incentivization, on-chain ticket distinction, model verification, and C2PA integration.
• Code Reviewers
  With an increase in developers contributing through bounties, we’re seeing a higher volume of pull requests, some of which are lower in quality. As the AI Networking team is also focused on delivering critical roadmap items, review time is limited, leading to potential delays in merging bounty contributions.
• Meeting with Studio Team
  We need to finalize the structure for the release. I already have working versions for TypeScript, Python, and Go.

Discussion Topics

• SDK Release
  Align release schedules and expectations across all parties, no deep dive required.
• Custom Containers and Generic API Interface
  Provide feedback on the implementation. We are already making significant progress.