search

bentoml / BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
https://bentoml.com
Apache License 2.0
7.13k stars 791 forks source link

BentoML roadmap overview #178

Closed parano closed 5 years ago

parano commented 5 years ago

This is living thread giving an overview of planned BentoML features on our roadmap - would love to hear your feedback. Join more discussion in our slack channel here: http://bit.ly/2N5IpbB

parano commented 5 years ago

Heroku deployment

Similar to the SageMaker and Serverless deployment BentoML currently provide, add support for Heroku platform

parano commented 5 years ago

Multi-model deployment workflow

Add support for easily creating and configuring ML services with multiple machine learning models

parano commented 5 years ago

Kukbeflow integration

Add support for deploying from Kubeflow project's training workflow

parano commented 5 years ago

Deployment Manager

A stateful server that tracks all your desired deployment state, deployment history and event logs. Allow users to interact via CLI, API and web UI, and talks to cloud platforms or kubernetes cluster for scheduling deployments.

parano commented 5 years ago

GPU support

Currently the BentoML generated docker images are not compatible with GPU environment and we are adding support for generating images that can utilize GPUs when serving a model

parano commented 5 years ago

TF-serving integration

Use tf-serving as tensorflow model backend, BentoML API server will handle REST API, request parsing, preprocessing and send GRPC to tf-serving for inferencing with/without GPU

parano commented 5 years ago

Closing in favor of the roadmap section in the upcoming BentoML guides

ALee008 commented 2 years ago

Hi @parano , you mention the roadmap section but I couldn't find it in the official documentation. Could you please provide a link to your official roadmap? Thank you.