Real-Time BitNet LLM Response on a Web Interface with Node.js and Socket.io

microsoft / BitNet

Official inference framework for 1-bit LLMs

MIT License

8.91k stars 567 forks source link

Real-Time BitNet LLM Response on a Web Interface with Node.js and Socket.io #59

Open stackblogger opened 20 hours ago

stackblogger commented 20 hours ago

A web interface designed for submitting queries and viewing real-time responses through a user-friendly UI. Built with Node.js for the frontend and a Python Socket server for backend processing, the system streams responses line by line in real time.

Enter your query, hit submit, and watch the output appear live as it's generated.

Repository link: BitNet.Js

Can this repo be mentioned in the Readme as Node.Js implementation of the cpp code?

alexeyvolkoff commented 20 hours ago

Does it keep context?

stackblogger commented 20 hours ago

I believe this functionality is not currently supported by the official code, but as new features are introduced, I will continue to enhance my repository to maintain compatibility and improve functionality