[Feature]: Allow running services and models without a gateway

[peterschmidt85]

Use case:

• As a user, I want to run a service/model and make it accessible to my project team.
• I don't want to set up a domain.
• I don't want to make the endpoint public.

Suggestion:

• In addition to the existing ability to create "managed" gateways (public IP address, custom domain), allow the dstack server to handle service/model traffic directly.

Motivation:

• So far, services have primarily been used for development, but the need to set up a domain and create a public endpoint poses a significant barrier. If the dstack server could run services/models out of the box, it would greatly simplify usage for both individuals and companies.

[peterschmidt85]

Another reason why this issue is important is that in our documentation, we often use tasks for deploying models because it doesn’t require creating a gateway, which is also not supported for all backends. However, when a model is running as a task, the user cannot use the UI to interact with the model.

[jvstme]

Implementation progress

• [ ] Python-based reverse proxy for services
  • [x] HTTP proxying
  • [x] Request/response streaming
  • [ ] Websocket proxying
  • [x] Proxying to multiple service replicas
  • [x] Auth
  • [ ] Configuring whether path prefix is stripped
• [x] Running services without a gateway
• [ ] OpenAI endpoint
• [ ] Collecting stats for replicas autoscaling
• [ ] Using the new proxy implementation on gateways
  • [ ] Nginx- and certbot- related logic
  • [ ] Installation and updates

Usage instructions for the current prototype

1. Run dstack server with the DSTACK_FF_PROXY=1 environment variable.
2. Make sure your project doesn't have a gateway or neither of the gateways is default.
3. Run a service.
4. Your service is now available at <dstack-server-base-url>/proxy/services/<project-name>/<run-name>/

[peterschmidt85]

Plus, ensure the UI can leverage the built-in proxy