Please note that API stability is not expected until 1.0 release. This update introduces breaking changes.
What's changed?
Documentation
Updated readme and documentation
API & SDK
Updated the format of model configuration yamls.
Backend
Completely replaced the text-generation-inference based backend with vLLM based backend. This means RayLLM now supports all models vLLM supports.
Improved observability and metrics.
Improved testing.
This update introduces breaking changes to model configuration YAMLs related to the new vLLM backend. Refer to the updated aviary/models/README.md file for details on the updated configuration file format.
In order to use Aviary, ensure you are using the official Docker image anyscale/aviary:latest.
Update RayLLM to v0.3.0.
Please note that API stability is not expected until 1.0 release. This update introduces breaking changes.
What's changed?
Documentation
API & SDK
Backend
This update introduces breaking changes to model configuration YAMLs related to the new vLLM backend. Refer to the updated
aviary/models/README.md
file for details on the updated configuration file format.In order to use Aviary, ensure you are using the official Docker image
anyscale/aviary:latest
.