[$$ BOUNTY] Add Phi-2 (2.7B) Model to TT-Buda Model Demos

Shubhamsaboo commented 4 months ago

Background:

TT-Buda, developed by Tenstorrent, is a growing collection of model demos showcasing the capabilities of AI models running on Tenstorrent hardware. These demonstrations cover a wide range of applications, aiming to provide insights and inspiration for developers and researchers interested in advanced AI implementations.

Bounty Objective:

We are excited to announce a bounty for contributing a new AI model demonstration to the TT-Buda repository. This is an opportunity for AI enthusiasts, researchers, and developers to showcase their skills, contribute to cutting-edge AI research, and earn rewards.

Task Details:

Integrate Phi-2 (2.7B) into the TT-Buda demonstrations.

Requirements:

The submission must include a comprehensive README.md detailing the model's architecture, implementation details, and usage instructions.
The model should be fully functional and tested on Tenstorrent hardware, ensuring compatibility and performance optimization.
Include sample inputs and outputs, demonstrating the model's capabilities.
Provide documentation on any dependencies and installation procedures.

Contribution Guidelines:

Fork the TT-Buda model demos repository.
Create a new directory within the model_demos folder following the naming convention: model_yourModelName.
Ensure your code adheres to the coding standards and guidelines provided in the repository's CONTRIBUTING.md file.
Submit a pull request with a detailed description of your model and any relevant information that will help reviewers understand and evaluate your contribution.

Evaluation Criteria:

Innovation and Relevance: How does the model contribute new ideas or solutions? Is it relevant to current challenges in AI?
Implementation Quality: Code readability, structure, and adherence to best practices.
Performance: Efficiency and performance on Tenstorrent hardware.
Documentation: Clarity and completeness of the accompanying documentation.

Rewards:

Contributions will be evaluated by the Tenstorrent team, and the best contribution will be eligible for $500 cash bounty.

Get Started with Grayskull DevKit

Dive into AI development with the Grayskull DevKit, your gateway to exploring Tenstorrent's hardware. Paired with TT-Buda and TT-Metalium software approaches, it offers a solid foundation for AI experimentation. Secure your kit here.

Connect on Discord

Join our Discord to talk AI, share your journey, and get support from the Tenstorrent community and team. Let's innovate together!

EwoutH commented 2 months ago

phi-3-mini is now released:

Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

The 3.8 billion parameter model should easily fit on the 8GB LPDDR4 if quantized to 8-bit. Maybe the bounty could be updated for Phi-3.

EwoutH commented 2 months ago

Phi-3-mini weights are released: https://huggingface.co/collections/microsoft/phi-3-6626e15e9585a200d2d761e3

edgerunnergit commented 2 months ago

hello, would love to give it a shot. could you please assign me to the issue? (both phi2 or phi3 works)

Shubhamsaboo commented 2 months ago

Sounds good, done @edgerunnergit!

I recommend you to check with the community members on Discord, if somebody is actually working on it. Would help you with the start or you can even tag team to solve this challenge.

EwoutH commented 2 months ago

@edgerunnergit do you have something to share at this point already?

edgerunnergit commented 2 months ago

@EwoutH I had a rough look but couldn't actually start yet. I would still love to give it a go. However I'm blocked till friday for final exams.

EwoutH commented 1 month ago

Microsoft released some new Phi-3 models!

The vision model is only 4.15B params so that should be able to run on 8GB Grayskull cards when quantized to 8 bit.

Medium is likely too large (14B), but small (7.39B) might fit using the block floating point format Grayskull supports, BFP4 (see also #59).

mvkvc commented 1 month ago

Got my Grayskull e75 this week, started working on phi-3-mini-4k and will share any progress.

tenstorrent / tt-buda-demos