Process Drosophila_3D+t stack in parallel using Azure Batch Python API

[wolny]

Processes the 5D tensor containing the stack of Drosophila embryo in parallel with ilasitk running on Azure Batch via Python API. The app uploads the stack into a single storage container and then segments the nuclei via a parallel workload running 50 ilastik processes (there are 50 time points in the stack, it's one segmentation task per time point) in a headless mode (see http://ilastik.org/documentation/basics/headless from more information). The output of each segmentation task (one tiff file per time point containing segmented 3D stack) is uploaded to a single storage container and then downloaded into the current working dir.

Prerequisites

• python 3.6
• pip3

Run the app

• Install virtualenv pip3 install virtualenv
• Go into the python dir and create virtual env for the app cd python && python3 -m virtualenv env
• Install required packages: pip install -r requirements.txt
• Update the Batch and storage account credentials inside the ilasik_azure_batch_client.py
• Run the script: python ilasik_azure_batch_client.py
• Wait for the script to finish and check that the output segmentation files have been successfully downloaded into your current directory (the files should be named drosophila_00-49_{t}_seg.tiff, where t corresponds to a given time point in the stack).

Sample segmentation output

Output of the ilastik segmentation task for a sample time point:

[lmiroslaw]

Hi Adam,

What a great contribution! How have you tested it ? Do you have Azure account?

Could you please submit a pull request to this repo? https://github.com/azurebigcompute/BigComputeLabs/tree/master/Microscopy_Lab

This is an official repo from our HPC team. This way we will have much larger exposure to the community.

Best, Lukasz

[wolny]

Hi Lukasz,

thanks for reviewing and merging the PR. Yes I've tested the script using a trial Azure account.

Happy to submit the same PR to the suggested repo.

Cheers, Adrian

[wolny]

@lmiroslaw please have a look at https://github.com/azurebigcompute/BigComputeLabs/pull/18

Cheers, Adrian