search

rustformers / llm

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models
https://docs.rs/llm/latest/llm/
Apache License 2.0
6.07k stars 355 forks source link

Api Server Example #414

Open e253 opened 1 year ago

e253 commented 1 year ago

I made an example rocket.rs api server _https://github.com/aurelianai/aurelian_server_ if anyone is interested.

pixelspark commented 1 year ago

I might have a more elaborate project utilizing rustformers/llm for a server that could be open sourced. Let me know if there is interest. It exposes WebSocket/SSE interfaces as well as endpoints for embedding, configurable sets of prompts and more.

e253 commented 1 year ago

Would love to see it and contribute!

pixelspark commented 1 year ago

Would love to see it and contribute!

It took a bit longer than expected, but here it is: https://github.com/pixelspark/poly 🎉 let me know what you think @e253

pixelspark commented 1 year ago

@philpax you might like it too

philpax commented 11 months ago

That's awesome! Sorry, I was very behind on my emails. I've added it to the README - shame I couldn't mention it at the talk.