Define the command-line interface for each step

[dchaley]

In deepcell-imaging#233 we separated the Python predict function into 3 independent functions.

Now we need to define the command-line interface. Batch jobs don't call Python functions: they invoke Python scripts, with command-line arguments.

For example we need to pass a numpy path on cloud storage, rather than a numpy array in Python.

[dchaley]

Input / output per step:

Preprocessing

def preprocess_image(model_input_shape, image, image_mpp):

Input:

• model_input_shape is a tuple, in our case (None, 256, 256, 2). Aka: the model was trained on 256x256 images using 2 channels.
• image: numpy array of shape (1, 512, 512, 2). Aka: one image size 512x512 with 2 channels.
• image_mpp: optional float, default None meaning use the model's micron per pixel.

Output:

• Numpy array of size (results, resized_rows, resized_cols, channels). The input will be resized if its mpp is not equal to the model's mpp (=0.5).

Infer

def infer(model, image, batch_size):

Input:

• model is a tensorflow.keras model loaded from file.
• image is a numpy array of shape (1, 512, 512, 2). Aka: one image size 512x512 with 2 channels.
• batch_size is a whole number, indicating how many files to predict with TF at once.

Output:

• dictionary: { 'whole-cell' : list of 2 arrays of shape (1, 512, 512, 1), 'nuclear' : list of 2 arrays of shape (1, 512, 512, 1) }

Postprocess

def postprocess(output_images, input_shape, compartment="whole-cell", whole_cell_kwargs={}, nuclear_kwargs={}):

Input:

• output_images : See infer output above
• input_shape : tuple, in our case (1, 512, 512, 2)
• compartment : string, either whole-cell, nuclear, both
• whole_cell_kwargs : key/val dictionary
• nuclear_kwargs : key/val dictionary

Output:

• Numpy array of shape (1, 512, 512, 1). Result number, rows, cols, segment number.

[dchaley]

Interfaces:

Preprocess:

Input:

• model_id : string: model id. Process looks up model input shape. (Hack for now: hard code!)
• image_uri : string: URI to input image npz file. (Eg gs://bucket/file.npz) Array name: image
• image_mpp : string representing a float, or blank / missing for none.

Output:

• preprocessed_image_uri : specify URI to write preprocessed image npz. Array name: image

Infer

Input:

• model_id : string: model id. Process looks up model URI by id. (Hack for now: hard code!)
• image_uri : string: URI to pre-processed image npz file. Array name: image
• batch_size : string representing a number, or blank/missing for default.

Output:

• model_output_uri : string: URI to write model outputs npz. Array names: arr_0, arr_1, arr_2, arr_3. Items 0 and 1 correspond to whole-cell. Items 2 and 3 correspond to nuclear.

Postprocess

Input:

• model_output_uri : string: URI for model outputs, see above.
• input_shape : string representing number of rows and columns as a list of numbers eg 512 512
• compartment : string: one of whole-cell, nuclear, both
• whole_cell_kwargs : string: list of key-val pairs e.g. key1 0.1 key2 1.5. Hack for now: leave blank, use defaults.
• nuclear_kwargs : string: list of key-val pairs e.g. key1 0.1 key2 1.5. Hack for now: leave blank, use defaults.

Output:

• predicted_segments_uri : string: URI to write npz file of predicted segments. Array name: image