should have top_k, top_p, temperature, repetition_penalty, max_new_tokens, max_time, num_return_sequences, and do_sample should all be passed in the "parameters" object instead. Additionally, it would be really great if you could add support to set the return_full_text parameter to false.
Hello!
I've been playing around with the interface to Hugging Face and noticed that the parameters are not being passed correctly to my endpoint. This code:
body: { inputs: prompt, top_k: this.settings.topK, top_p: this.settings.topP, temperature: this.settings.temperature, repetition_penalty: this.settings.repetitionPenalty, max_new_tokens: this.settings.maxGenerationTokens, max_time: this.settings.maxTime, num_return_sequences: this.settings.numberOfGenerations, do_sample: this.settings.doSample, options: { use_cache: true, wait_for_model: true, }, },should have top_k, top_p, temperature, repetition_penalty, max_new_tokens, max_time, num_return_sequences, and do_sample should all be passed in the "parameters" object instead. Additionally, it would be really great if you could add support to set the return_full_text parameter to false.
Reference: https://huggingface.co/docs/api-inference/en/detailed_parameters
Thank you!