document setup which works with oplateks text-generation-webui

kasnerz / quintd

Data-to-text generation with LLMs using ad-hoc datasets.

https://d2t-llm.github.io

MIT License

8 stars 2 forks source link

document setup which works with oplateks text-generation-webui #4

Closed oplatek closed 4 months ago

oplatek commented 4 months ago

gpu-python --gpu_mem 48 server.py --settings settings_local.yaml --listen --listen-host 0.0.0.0 --api --model "mistralai_Mistral-7B-Instruct-v0.1"

@kasnerz I added dependencies and commit your changes.

Feel free to close the request and commit your changes from our cluster, but please add the requirements

kasnerz commented 4 months ago

Thanks! All the discrepancies were caused by the quick development of text-generation-webui. Their original API got deprecated in favor of OpenAI-like API.

As a hotfix, I linked to the respective commit of text-generation-webui (still using the original API) in the README.

But since than I came to realize that it is better to have a code that everyone can run instead of an exact replica of my experimental code. I will integrate the changes.