🚀 Overview
This PR introduces a series of improvements aimed at enhancing user experience and refining the codebase. Here's a breakdown of the changes:
⚡ 1. Optimized performance: llama.cpp & exllama
Made performance improvements by changing the text generation logic.
🌐 2. Tunnel through Cloudflare
Expose this API to the external network using the --tunnel option.
⚙️ 3. CLI args Refinement
Moved argparse.ArgumentParser to config.py.
🐞 4. Bugfix: niceness of process
Fixed a bug where the niceness of the process couldn't be modified in a docker environment.
🔜 5. Enhancement: required option in function call schema
The function call feature is not yet implemented. Stay tuned!
🚀 Overview This PR introduces a series of improvements aimed at enhancing user experience and refining the codebase. Here's a breakdown of the changes:
⚡ 1. Optimized performance: llama.cpp & exllama
🌐 2. Tunnel through Cloudflare
--tunneloption.⚙️ 3. CLI args Refinement
argparse.ArgumentParsertoconfig.py.🐞 4. Bugfix: niceness of process
🔜 5. Enhancement:
requiredoption in function call schema