Closed simonw closed 10 months ago
Got this working:
llm -m gguf \
-o path ~/Downloads/una-cybertron-7b-v2-bf16.Q8_0.gguf
'Instruction: Write a poem about an otter
Response:'
Which outputs:
Otters are elusive, playful creatures Swift in the water and crafty in their features Hiding beneath kelp or drifting on logs With a smile that stretches wide beyond our dogs
They dance gracefully underwater, chasing fish Astonishing us all with such splendid bliss Otters are known for their love of toys Balls are tossed into the water, where they joyously play
These marvelous mammals leave an everlasting impact Their presence bringing life to our waters and rivers unquenched Dazzling spectacles for all who watch A magical moment for some fortunate souls to catch
Otters embody a spirit unbounded by boundaries Inspiring us to embrace the beauty of nature's wonders Leaving footprints on our hearts that will always be etched With lessons from these graceful, otter-ly loved creatures
I realized it would be really convenient if you could execute a new GGML file without having to first do the registration dance.
I could do this with a model called
ggml
that takes a requiredpath
option.