search

Atinoda / text-generation-webui-docker

Docker variants of oobabooga's text-generation-webui, including pre-built images.
GNU Affero General Public License v3.0
365 stars 77 forks source link

make settings.yaml persistent!!! #40

Open Gee1111 opened 5 months ago

Gee1111 commented 5 months ago

find a way to make it persistent. e.g. apply a patch after u get https://github.com/oobabooga/text-generation-webui.git in dockerfile to change the default dir for settings.yaml to eg /extensions

it's absolutely annoying

heres the patch u have to use:

https://github.com/Gee1111/text-generation-webui/blob/main/make_settings_persistent.patch

edit send u a merge request but its better to put the make_settings_persistent.patch in your own repo and access it from there

Atinoda commented 5 months ago

TLDR; - ./config/settings.yaml:/app/settings.yaml

Hi, thanks for sharing your issue - and big thanks for submitting a PR too!

You prompted me to look into this a bit... normally Docker images are stateless - in this case, that means the settings should be configured via EXTRA_LAUNCH_ARGS string and not by fiddling around in the runtime. However, it seems that extensions and modules make liberal use of the settings.yaml file, and I don't think all options are settable via flags - so your suggestion makes sense in the context of this project.

During the course of examining the upstream code, I noted that there is an option to specify a settings file using the --settings $FILE launch argument. I would prefer to avoid patching the project source code unless absolutely unavoidable, so for that reason I will not merge your PR as it stands because it relies upon changes to the application source. However, if you would like to develop an approach that does not patch the source, I would consider merging it!

I do not have an exact implementation for this feature yet, but two approaches come to mind: 1) symlink the default file to a mountable folder and add business logic to the entrypoint script to handle its initialisation, or 2) specify a custom file using the setting flag I mentioned earlier (although I do not 100% like this one because it relies on passing an argument to the program, which a user may override).

For an immediate fix, I suggest just mounting a settings.yaml directly to the correct location by adding -./config/settings.yaml:/app/settings.yaml as a new entry in the volumes section of docker-compose.yml. Note that you may need to instantiate that file first, either with a template or by extracting settings from a configured container.

Gee1111 commented 5 months ago

found this "--settings $FILE" too but this dosnt work

update: nevermind i got it working