RayLLM v0.3.0

[Yard1]

Update RayLLM to v0.3.0.

Please note that API stability is not expected until 1.0 release. This update introduces breaking changes.

What's changed?

• Documentation

  • Updated readme and documentation

• API & SDK

  • Updated the format of model configuration yamls.

• Backend

  • Completely replaced the text-generation-inference based backend with vLLM based backend. This means RayLLM now supports all models vLLM supports.
  • Improved observability and metrics.
  • Improved testing.

This update introduces breaking changes to model configuration YAMLs related to the new vLLM backend. Refer to the updated aviary/models/README.md file for details on the updated configuration file format.

In order to use Aviary, ensure you are using the official Docker image anyscale/aviary:latest.

[Yard1]

@richardliaw done