Sets up reginald/models dir which contains main.py script to set up API for llm. Script is run using poetry run uvicorn main:app and sets up llm by calling setup_bot.py and using either env variables or defaults for choice of model etc.
Fixes the 'direct_message' and 'channel_mention' parts of llama cpp model (we can change all to respond in future PR).
Sets up reginald/bot dir which contains setup_bot.py to set up the slack bot. Bot now takes 'api_urlargument instead ofmodel` argument and makes call to API instead of call to model when getting response.
Still to do (potentially different PR):
[x] Re-add run.py for all-in-one set up
[x] Re-add command line options for set_up llm
[x] Re-add path option for llama cpp
[x] Check queuing still works - i think the requests.get() needs ot be asyncronous.!!
This PR:
reginald/models
dir which containsmain.py
script to set up API for llm. Script is run usingpoetry run uvicorn main:app
and sets up llm by callingsetup_bot.py
and using either env variables or defaults for choice of model etc.reginald/bot
dir which containssetup_bot.py
to set up the slack bot. Bot now takes 'api_urlargument instead of
model` argument and makes call to API instead of call to model when getting response.Still to do (potentially different PR):