search

Blaizzy / fastmlx

FastMLX is a high performance production ready API to host MLX models.
Other
159 stars 12 forks source link

Add support for token streaming, parallel jobs and custom CORS #4

Closed Blaizzy closed 2 months ago

Blaizzy commented 2 months ago

This PR adds:

  1. Multi-modal token streaming.
  2. Support for Parallel calls (single and multiple models) by default upto N workers.
  3. Supported model type endpoint.
  4. Delete model endpoint.
  5. Custome CORS.

Todo:

Closes #2, Closes #5