googlegenomics / gcp-deepvariant-runner

This repository contains a docker container that runs DeepVariant on the Google Cloud Platform.
Apache License 2.0
2 stars 6 forks source link

Starting multiple gcsfuse processes using PAPI action list #14

Closed samanvp closed 5 years ago

samanvp commented 5 years ago

In the latest version of DeepVariant (0.8.0) gcsfuse binary has been removed from the docker image. In order to be able to run make_example on multi core machines we still need to start multiple gcsfuse processes (one per core). We achieve this by launching PAPI using a json file that includes action list in the following order:

This will be essentially equivalent to what we were doing using DeepVariant 0.7.2

samanvp commented 5 years ago

I ran a bunch of profiling experiments to compare this new implementation of gcsfuse to our current one (which gcsfuse binary exists in deepvariant docker image) and the performance is almost identical.