Closed FoxxMD closed 4 months ago
Heya, I reverted the dependabot PR, but this looks like a better solution to me. I also have never used /generate
. I was like, "I don't remember ever adding that", but it turns out it was someone's PR that I merged
Makes the following changes:
Fix python version
Python fails when using 3.12, downgrade to 3.11 to fix this.
Move Requirements Installation to Runtime
The addition of
/generate
requirestransformers
andtorch
which adds 5GB+ to thesite-packages
directory due to models and other resources that need to be downloaded. This isn't an issue per se but it does mean that docker images are now 6GB+ instead of the previous ~360MB since requirements install is done during image build.This PR moves requirements installation to
docker-entrypoint.sh
so that these packages are instead downloaded when the container first runs, making the image lightweight again. It also enables...Optional
/generate
inclusionAs the bot hoster, and a user, I don't personally need or use
/generate
. My users are perfectly happy with the prompts they generate organically! However I still have to incur the downloads, slower startup due to requirements install, and 10x increase in memory usage for the application to load torch/transformers.This PR adds the environmental variable
USE_GENERATE
as a string boolean (true
orfalse
). It defaults totrue
to maintain backward compatibility. Behavior:false
indocker_entrypoint
requirements_no_generate.txt
forpip install
to avoid downloading torch/transformerspython core/setup_generate.py
false
at bot startup skips loadinggeneratecog
Enabling me to use aiyabot without
/generate
and without the system requirements incurred by the needed packages.