End-To-End LivePortrait Implementation Bounty [$2500]

[rickstaa]

Overview

We're thrilled to announce the expansion of our Livepeer AI Network with the introduction of a new LivePortrait pipeline. This cutting-edge feature will enable users to animate still images into videos, adding a new dimension of creativity. 🔥

We’re seeking community assistance to implement this critical pipeline in the Livepeer AI-Network that includes both ai-worker and go-livepeer components. By contributing to this feature, you'll help enrich our existing video creation pipeline, while providing a more robust creative platform for artists and enthusiasts alike. 🚀

Required Skillset

• Proficiency with FastAPI.
• Strong experience with Python.
• Strong Go programming skills.
• Familiarity with Docker and Hugging Face 🤗.

Bounty Requirements

1. Implementation: Create a functional /live-portrait route and pipeline within the AI-worker repository. This new pipeline should be accessible through Docker on port 8900. Also, develop the necessary code within the go-livepeer repository to integrate access to the live-portrait pipeline from the ai-worker component. This includes implementing the payment logic and job routing to ensure seamless operation within the network.

2. Functionality: The pipeline must process an initial image and a video to generate an animated version of the image. It should also include the necessary preprocessing steps for the video as required by the model. Also, ensure that users can submit AI job requests to the network in a manner consistent with other AI-Network features.

   Example request: curl -X POST "https://your-gateway-url/live-portrait" -F model_id="" -F driver_video=@directory-path-to-video.mp4 -F initial_image=@directory-path-to-image.jpeg

Scope Exclusions

• NONE

Implementation Tips

To learn how to create a new pipeline, refer to the Hugging Face spaces. You can also explore the following pull requests to see how other pipelines were implemented:

• Pull Request #96
• Pull Request #103

In some cases, you may not be able to integrate the new pipeline directly into the regular AI Runner Docker container due to dependency conflicts or missing packages. If this occurs, you can follow the approach outlined in https://github.com/livepeer/ai-worker/pull/185 to create a custom container for the pipeline. This approach uses the regular AI runner as the base while keeping the base container lean.

Additionally, keep the following best practices in mind:

• Leverage Existing Resources: Explore existing implementations of LivePortrait on GitHub README and this Hugging Face space. These can serve as useful starting points.

• Use Developer Documentation: Our developer documentation for the worker and runner provides tips for mocking pipelines and direct debugging, which can streamline the development process. Similarly, developer documentation for the installation of go-livepeer and the general Livepeer documentation for example usage and setup instructions offer valuable insights that can expedite your development process, including automatic scripts for orchestrators and gateways, and direct debugging techniques.

• Update OpenAPI Specification: Run the runner/gen_openapi.py script to generate an updated OpenAPI specification.

• Generate Go-Livepeer Bindings: In the main repository directory, execute the make command to generate the necessary bindings, ensuring compatibility with the go-livepeer repository.

• Build Binaries: Run the make command in the main repository folder to generate Livepeer binaries. This will allow you to test your implementation and ensure it integrates smoothly.

• Create Docker Images: Build Docker images of Livepeer and test them using appropriate tools and settings to identify any edge cases or bugs. This step is crucial for ensuring robustness and reliability in your implementation.

How to Apply

1. Express Interest: Comment on this issue to indicate your interest and explain why you are well-suited for this task.
2. Await Review: Our team will review the applications and choose the most qualified candidate.
3. Get Assigned: If selected, the GitHub issue will be assigned to you.
4. Start Working: Begin the task! If you need support or have any questions, comment on the issue or join discussions in the #developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue once your pull request is ready for review.
7. Receive Your Bounty: After your pull request is approved, we will arrange the bounty payment.
8. Earn Recognition: Your contributions will be highlighted in our project’s changelog.

We’re eager to see your interest in contributing to our project! 💛

[!WARNING] Please ensure the issue is assigned to you before starting work. To avoid duplication of efforts, unassigned issue submissions will not be accepted.

[JJassonn69]

Thank you @rickstaa.

[JJassonn69]

Go-livepeer side PR and ai-worker side PR are ready for final review.

This completes the bounty requirement for end to end implementation of LivePortrait on Livepeer AI Network.