Batch: Support long running jobs, including GPU workloads - Githubissues

nitrictech / nitric

Nitric is a multi-language framework for cloud applications with infrastructure from code.

https://nitric.io

Apache License 2.0

1.23k stars 52 forks source link

Batch: Support long running jobs, including GPU workloads #664

Closed tjholm closed 1 month ago

tjholm commented 3 months ago

Feature Request

Suggestion

Current serverless compute offerings supported by nitric have strict time limits.

AWS Lambda (15 minutes)
Google Cloud Run (60 minutes)
Azure Container Apps (4 minutes for HTTP ingress)

They also offer highly constrained vertical scaling of hardware, as they are systems designed with horizontal scaling in mind.

This is a proposal to introduce a new kind of Nitric compute offering that will look to take advantage of serverless HPC offerings in the cloud.

Value

There are HPC, Data and ML use cases that nitric is currently too hardware constrained to handle.

e.g.

Large ETL workloads
Big Data analysis workloads
ML Training and Finetuning
Other general workloads requiring access to GPUs
Other long-running tasks (hours, days, etc.)

jyecusch commented 1 month ago

Released to Preview