search

simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp
Apache License 2.0
136 stars 19 forks source link

Set max_tokens parameters to 4000 #18

Closed almet closed 11 months ago

almet commented 11 months ago

It avoids truncated answers. Also reordered the imports.

simonw commented 11 months ago

Thanks for figuring this out!

simonw commented 11 months ago

I'm going to use this as the default and add an option to change it.

simonw commented 11 months ago

Tested locally and it worked:

llm -m mi 'twenty amazing names for pet pelicans with fun descriptions'
  1. Splashy Sam - A playful and adventurous pelican with a love for diving into the water and splashing around.
  2. Seashell Sally - A sophisticated and elegant pelican with a passion for collecting seashells and exploring the ocean.
  3. Fliptastic Fred - A graceful and agile pelican with a talent for flipping in the air and performing acrobatics.
  4. Sandpaper Sammy - A rugged and resilient pelican with a strong beak and feathers that feel like sandpaper to the touch.
  5. Bubbles Benny - A friendly and outgoing pelican with a love for chasing bubbles and playing in the water.
  6. Crunchy Critter - A curious and adventurous pelican with a taste for crunchy treats like fish and shrimp.
  7. Sunny Suzie - A cheerful and energetic pelican with a passion for sunbathing and soaking up the rays of the sun.
  8. Waddle-Waddle Wendy - A sweet and gentle pelican with a waddling gait and a love for napping in the sun.
  9. Scuba Sally - An adventurous and daring pelican with a passion for exploring the depths of the ocean in search of new sights and treasures.
  10. Beach Babe Bella - A stylish and fashionable pelican with a love for lounging on the beach and soaking up the sun.
  11. Swimming Sam - A strong and powerful pelican with a talent for swimming and diving into the water with ease.
  12. Munchkin Max - A tiny and adorable pelican with a big personality and a love for snacking on tasty treats.
  13. Snap-Happy Sally - An energetic and fun-loving pelican with a passion for capturing memories through photographs and videos.
  14. Flockster Freddy - A social and outgoing pelican with a talent for organizing and coordinating group activities with friends and family.
  15. Dancing Daisy - A graceful and elegant pelican with a love for music and dance, often seen twirling and swaying to the rhythm.
  16. Splendid Samantha - A sophisticated and refined pelican with a passion for art and creativity, often seen painting and drawing beautiful scenes of nature and wildlife.
  17. Seashell Spotter Simba - An adventurous and curious pelican with a love for searching for new and interesting seashells on the beach.
  18. Tasty Treasure Hunter Tommy - A foodie pelican with a passion for exploring new flavors and dishes, always searching for the best tasting treats to savor.
  19. Feathered Friend Foster - A kind and gentle pelican with a love for nurturing and caring for others, often seen helping those in need or offering a comforting embrace.
  20. Watery Wonderland Wendy - An adventurous and curious pelican with a passion for exploring new and exciting waterways, always eager to discover new sights and sounds along the way

That's using the new Mistral Instruct, installed like this:

llm llama-cpp download-model \
  https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q8_0.gguf \
  --llama2-chat -a mi -a mistral-instruct
simonw commented 11 months ago

Released in https://github.com/simonw/llm-llama-cpp/releases/tag/0.2b1

almet commented 11 months ago

Thanks :-)