search

kvcache-ai / ktransformers

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Apache License 2.0
654 stars 31 forks source link

using docker start api server can't set max_new_tokens #31

Open goldenquant opened 1 month ago

goldenquant commented 1 month ago

using docker start api server can't set max_new_tokens

UnicornChan commented 1 month ago

In the past few days, we have been focusing on enhancing support for native Windows, multiple GPUs. About making improvements to the server and Docker. These updates will be released in the next version.