search

bottlerocket-os / bottlerocket

An operating system designed for hosting containers
https://bottlerocket.dev
Other
8.4k stars 498 forks source link

[ECS] Inferentia support #1075

Open samuelkarp opened 3 years ago

samuelkarp commented 3 years ago

This issue is tracking adding support for Inferentia devices to the ECS variant of Bottlerocket.

This will require adding the Neuron driver and runtime.

GSvensk commented 3 years ago

Does this apply to EKS as well?

zmrow commented 3 years ago

Does this apply to EKS as well?

This issue is to track support for the ECS variant. I'm not sure if EKS would work the same way but we'd probably want to track them separately.

stmcginnis commented 1 year ago

Is this still needed?

prashant-prodigal commented 9 months ago

are we taking this up in next release @stmcginnis ? Do we have an expected timeline for this? We plan to use this heavily.

stmcginnis commented 9 months ago

Sorry for the delay!

No, this will not be included in the next release, unfortunately. There isn't a timeline currently, but it's good to have comments like your's to help gauge interest and have a little extra data to back prioritization.

Contributions are welcome if anyone wants to take a crack at adding it, but until then this is on the list, but not currently assigned anyone to work on it.

AWS-tomofun commented 8 months ago

This is a comment to recommend this issue.

Demand for AI/ML has increased, and requests from users to use GPU or Inferentia chip on ECS are increasing. There is a plan to support GPU in Fargate, and I think the options for using GPU in ECS will expand for them. However, since the Inferentia chip doesn't support either EC2 or Fargate, I think this issue is one solution that supports a variety of use cases like these.

prashant-prodigal commented 7 months ago

Hey, any progress on this @AWS-tomofun @stmcginnis ?