Implement OneDiff optimization on T2I pipeline [50 LPT]

[rickstaa]

Overview

Following a successful research bounty into the OneDiff optimization algorithm, we've discovered that combining OneDiff with OneFlow can accelerate inference by approximately 60% while maintaining image quality. The only drawback is an increase in startup time from 2 seconds to 68 seconds. Given that a 60% reduction in inference time can significantly enhance network performance, we believe it's crucial to provide orchestrators with the means to enable this optimization.

Currently, the core AI team is occupied with major restructuring tasks, including the AI worker and pipeline generalisations, developing new pipelines for startups, and releasing client SDKs. Therefore, we are contacting the community to implement this vital optimization for us 🏗️⚡.

Required Skillset

• Strong proficiency in Docker.
• Advanced Python programming skills.
• Expertise in managing Python virtual environments.
• Experience with the Hugging Face Diffusers library.

Bounty Requirements

1. Enable OneDiff Optimization:
   • Implement a mechanism for orchestrators to enable the OneDiff optimization with the OneFlow compiler on the T2I pipeline using a ONEDIFF Environment variable, similar to the approach used for Deep Cache and Stable Fast optimizations. For reference, see the implementation here.
2. Integration in AI-Runner Container:
   • Ensure that the OneDiff optimization can be integrated into the ai-runner container without causing dependency conflicts with existing pipelines or optimizations. This will likely involve updating Torch.

Implementation Tips

• Utilize Developer Documentation:
  • Refer to our developer documentation for both the worker and runner. These resources offer valuable tips for accelerating your development process, including how to mock pipelines and enable direct debugging.
• Review Existing Optimizations:
  • The T2I pipeline already includes several optimizations. Review the existing code to understand these optimizations and incorporate similar strategies when implementing the new OneDiff optimization.
• Seek Help Promptly:
  • This bounty is challenging and may involve dependency errors that the core AI team has likely encountered before. If you get stuck, don’t hesitate to ask questions in the #🛋│developer-lounge channel.

How to Apply

1. Express Your Interest: Comment on this issue to indicate your interest and explain why you're the ideal candidate for the task.
2. Wait for Review: Our team will review expressions of interest and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Dive into your task! If you need assistance or guidance, comment on the issue or join the discussions in the #🛋│developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on this GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: We'll arrange the bounty payment once your pull request is approved.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Thank you for your interest in contributing to our project 💛!

[!WARNING] Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

[lukiod]

Hey i am interested in this bounty.

[rickstaa]

@lukiod, it's great to see that you've decided to take on the implementation following the resolution of https://github.com/livepeer/bounties/issues/31 🚀. Given your experience with the OneDiff optimization in your research, I believe you're the ideal candidate for this task. I've assigned this bounty to you accordingly.

[RaghavArora14]

Hi, I am interested if this bounty is still open can I be assigned to this bounty?