search

CloudLLM-ai / cloudllm

CloudLLM is a Rust library designed to seamlessly bridge applications with remote Language Learning Models (LLMs) across various platforms.
MIT License
5 stars 1 forks source link

Very nice work! #1

Open arvidjohansen opened 3 months ago

arvidjohansen commented 3 months ago

Hi,

I just found this repo by chance when looking for something else..

..and at the moment I don't have the capacity to sit down and learn rust...

But if that ever changes I'm definitely coming back to take a look at this repo.

Code and project looks nice and clean, and the idea behind is ofc very good!

So if I ever change my mind, I'll definitely be back!

Good luck.

gubatron commented 3 months ago

Thank you so much!

I've been working on a lot of other stuff in the meantime and during this period the OpenAI API has finally included a way to have a "conversation id" or "thread id" which now makes it a lot better.

The current API keeps track of the entire chat conversation and sends the whole thing every time, thus costing a lot more in tokens.

This will be the next update, if you get up to speed with rust, would love to receive pull requests.

Thanks for the compliments, hope you find it useful!

arvidjohansen commented 3 months ago

Aha yeah that makes sense!

I'm trying to keep track of the constantly evolving AI/LLM-backends myself, it's evolving quite rapidly indeed :)

I would love to contribute!

Perhaps it's not such a faint idea after all - based on your response we have a lot of common problems to solve, except that I'm using (mostly) Python.

I've been a developer for many years working mostly with web/python/backend/api stuff., with alot of experience with system administration, OS, networking, linux, hosting and all that, but only recently got into AI.

So now I'm trying to figure it all out with different prompt formats, responses, different backends etc etc to build something really cool.

How about setting up a discord-server for the community?

I'd be happy to help you with that or anything else to bring thinking heads closer together.

Looking forward to hear from you! -Arvid

gubatron commented 3 months ago

let me get back from vacation in about 2 weeks, I want to get the API first to reflect the more efficient way to interact with OpenAI and then I want to have a Groq API client and if possible an Anthropic/Claude one as well, then I think we can start thinking of having a community, so far you'd be the first person to ever contact me about this work.

I've also done a ton of python development in the past, mostly backend and lots of command line tools, and at some point even Desktop development with PyQt (Qt for Python), it's an addictive programming language