search

ohsu-comp-bio / funnel

Funnel is a toolkit for distributed task execution via a simple, standard API.
https://ohsu-comp-bio.github.io/funnel
MIT License
121 stars 32 forks source link

Using instances with GPUs #664

Open kmavrommatis opened 2 years ago

kmavrommatis commented 2 years ago

Hi, apologies if I have missed this in the documentation. Is there aw way to make funnel start a docker container using the --gpu xx or --gpu all argument (xx=number of GPUs to use)? I am passing jobs to an AWS batch queue with gpu enabled instances, but the task jobs don't seem to get the information that the GPUs are available on the node.

Thanks in advance for your help

kellrott commented 2 years ago

This isn't in funnel yet. We're looking at adding backend_parameters into the TES specification ( https://github.com/ga4gh/task-execution-schemas/pull/154 ). Once this is available, we will find a way to configure mappings into Funnel.

obonyojimmy commented 1 year ago

Quick question, is this implementated yet ?

kellrott commented 1 year ago

The TES 1.1 specification has been officially released. My team is working on figuring out when we can schedule some of our engineering time to work on upgrading Funnel.
So, no it is not implemented yet. But we hope to get it done soon.