This stuff is great, I'm having a ton of fun working on it -- thanks for putting it all together! It's so cool to see it working on a consumer-grade GPU!
PR:
Adds a Dockerfile for easy use of the 7bn parameter model.
Adds a gif to the README and a few sentences about the model.
Adds a line autograd_4bit.py that was causing the UI to fail for me. I'm not quite sure what is going on there.
Seems like you're moving super fast and most people here are interested in training, so if you're not interested in dealing with things only for inference / easy of use, feel free to just close the PR.
This stuff is great, I'm having a ton of fun working on it -- thanks for putting it all together! It's so cool to see it working on a consumer-grade GPU!
PR:
Seems like you're moving super fast and most people here are interested in training, so if you're not interested in dealing with things only for inference / easy of use, feel free to just close the PR.