search

vikhyat / moondream

tiny vision language model
https://moondream.ai
Apache License 2.0
4.48k stars 393 forks source link

Response cut off early, still very good though #44

Open duracell80 opened 5 months ago

duracell80 commented 5 months ago

Excellent speed improvements with this, thank you for the great work!

I noticed quite a few results are being cut off mid sentence. Using the image at the top of this article, the description gets cut off early.

"The image features a modern and well-lit home office with a large desk situated in the center of the room. The desk is equipped with a computer monitor, keyboard, and mouse, creating a functional workspace. A chair is placed in front of the desk, providing a comfortable seating option for the user.

In addition to the main desk, there is a bookshelf filled with various books, adding a touch of organization and intellectual ambiance to the room. A potted plant is also present, adding a touch of greenery and life to the office.

A chair is placed in front of the desk, and a"

vikhyat commented 5 months ago

There's a max token limit of 128 here, maybe we should bump it up.

https://github.com/vikhyat/moondream/blob/main/moondream/moondream.py#L55

duracell80 commented 5 months ago

It could be a parameter like --prompt and use the current as default? Like --chars=

Or something like splitting the result and if the last sentence runs over the token limit, then try not to print out anything after the last detected period / full stop.

vikhyat commented 4 months ago

Bumped it up to 256: https://github.com/vikhyat/moondream/blob/main/moondream/moondream.py#L98

Can go up further if it still keeps cutting off.