Add Text Generation Inference with JSON output

MaartenGr / KeyBERT

Minimal keyword extraction with BERT

https://MaartenGr.github.io/KeyBERT/

MIT License

3.31k stars 336 forks source link

Add Text Generation Inference with JSON output #235

Closed joaomsimoes closed 1 week ago

joaomsimoes commented 2 weeks ago

Added the option to use TGI from HuggingFace using JSON format. It makes the output more predictable. Also it is useful to use TGI in case we use it for other use cases.

joaomsimoes commented 1 week ago

I also added the inference_kwargs to the extract_keywords function. I believe it is good to have control of temperature and max new tokens during inference, in case we have two different tasks that require a bit more control.

MaartenGr commented 1 week ago

Awesome, everything looks good to me! Thank you for the work on this it is highly appreciated 😄