[Feature]: Initial support for routers (using AWS Bedrock)

[jvstme]

Problem

Some users need to work with both custom and off-the-shelf language models. Deploying off-the-shelf models with dstack may be less convenient and financially viable than using platforms such as AWS Bedrock or Vertex AI, which provide models as a service. In addition, some proprietary models are only available through MaaS platforms. This results in users having to switch back and forth between dstack and MaaS platforms.

Solution

Add support for MaaS platforms starting with AWS Bedrock. Add new router configurations that will allow Bedrock models to be exposed through the dstack-gateway OpenAI-compatible API.

Workaround

Users can use MaaS platforms directly or use proxy solutions to bring MaaS models and models deployed by dstack into a single interface.

Implementation Steps

• [ ] Storing data about models in dstack-gateway
• [ ] Internal API for model management in dstack-gateway
• [ ] Bedrock implementation in dstack-gateway
• [ ] MaaS-specific settings in backend configuration
• [ ] Storing data about models and routers in dstack-server
• [ ] dstack-server routers API
• [ ] Routers Python API
• [ ] dstack apply router configurations
• [ ] dstack router list and dstack router delete commands
• [ ] Documentation

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale. Please reopen the issue if it is still relevant.