phidatahq / phidata

Build AI Agents with memory, knowledge, tools and reasoning. Chat with them using a beautiful Agent UI.

https://docs.phidata.com

Mozilla Public License 2.0

15.63k stars 2.15k forks source link

Cloudflare AI workers <> Feature/cloudflare ai workers #1371

Closed vineetbin closed 2 weeks ago

vineetbin commented 1 month ago

Cloudflare AI Worker Integration

Overview

Edge-based integration for running open-source AI models using Cloudflare Workers. Currently supports LLAMA_3_8B_INSTRUCT with free tier access.

Key Features

Cost Efficient: Utilizes Cloudflare's free tier
Easy Scaling: Add more models based on demand
Edge Computing: Low latency, globally distributed

Flexible: Supports multiple open-source models

Implementation

Current Model → LLAMA_3_8B_INSTRUCT
Deployment    → Cloudflare Workers
Tier          → Free

manthanguptaa commented 3 weeks ago

Hey @vineetbin, an excellent addition to phidata! But we will be moving from Assistants and llm starting from the 3.0.0 version. I had suggest to port this to the agent model. Let me know if you have any questions. Also don't forget to add a cookbook as well.

manthanguptaa commented 2 weeks ago

@vineetbin are you still working on this?

vineetbin commented 2 weeks ago

@vineetbin are you still working on this?

Oh No Manthan, I spent some time doing it, but it was only a make do version for getting the Interview call. If you guys want I can do further additions too and complete it.