search

Blaizzy / fastmlx

FastMLX is a high performance production ready API to host MLX models.
Other
159 stars 12 forks source link

max_tokens not overriding the default #5

Closed stewartugelow closed 2 months ago

stewartugelow commented 2 months ago

I'm only getting 100 tokens, no matter what I pass in the request.

Blaizzy commented 2 months ago

Hey @stewartugelow

Thanks for pointing it out!

I noticed during a stream of user, and should be fixed on the next release #4