Deploy model

[covasquezv]

Hi! Just to confirm, is MIScnn compatible with tensorflow-serving for model deployment? Thanks in advance!

[muellerdo]

Hello @covasquezv,

good question. To be honest, I don't know because I have never worked with TF Serving.

But after reading some documentation, I'm quite sure that it has to be compatible. To simplify things, MIScnn is just a framework for efficient batch generation and preprocessing methods which are required for medical image segmentation. But the backbone is still a Tensorflow-Keras model.

And TF Serving is able to handle Tensorflow-Keras models, so you just have to implement your desired API endpoints and you are good to go.

Check out this: https://www.tensorflow.org/tfx/guide/keras

But how to implement the API endpoints with MIScnn is another question. It is probably the best way to utilize the keras model itself for the TF Serving and completely avoiding the MIScnn Neural_Network class (which acts normally as the runner of the pipeline). I would start tying to copy the Neural_Network class code for initialization, training and prediction in the associated API endpoints of TF serving.

Maybe it is also possible to pass a MIScnn Neural_Network instance instead of the Keras model itself, but I suspect that TF serving is doing some background stuff with the model (e.g. log output, architecture parameter analysis etc). Therefore, I would highly recommend not to deviate from the documentation.

Also check out this great blog post: https://blog.tensorflow.org/2018/08/training-and-serving-ml-models-with-tf-keras.html

Cheers, Dominik

[covasquezv]

Hey, I forgot to update you. I was able to successfully deploy the model. I saved the weights following https://www.tensorflow.org/tfx/guide/keras. Then using the docker tensorflow-serving I leave it running in a server. To build and run the docker I used the following command: sudo docker run --runtime=nvidia -i -p 8501:8501 --rm=True --name=miscnn_fase2 -v "/home/ubuntu/segmentation/app/models/miscnn/:/models/miscnn/" -e MODEL_NAME=miscnn -t tensorflow/serving:latest-gpu

Since in my case, the model works with patches I had to write a script based on the MIScnn preprocessing, predict and postprocessing as follows:

from miscnn.data_loading.interfaces import NIFTI_interface
from miscnn import Data_IO, Preprocessor
from miscnn.neural_network.data_generator import DataGenerator
from miscnn.processing.subfunctions import Normalization, Clipping, Resampling
import requests, json

interface = NIFTI_interface(channels=1, classes=3)

data_io = Data_IO(interface, input_path=input_path, delete_batchDir=False)
sample_list = data_io.get_indiceslist()
sample_list.sort()

sf_clipping = Clipping(min=-1250, max=250)
sf_normalize = Normalization(mode="grayscale")
sf_resample = Resampling((1.58, 1.58, 2.70))
sf_zscore = Normalization(mode="z-score")
sf = [sf_clipping, sf_normalize, sf_resample, sf_zscore]

 preprocessor = Preprocessor(data_io, data_aug=None, batch_size=1, subfunctions=sf,
                                prepare_subfunctions=True, prepare_batches=False,
                                analysis="patchwise-crop", patch_shape=(160, 160, 80))

preprocessor.patchwise_overlap = (80, 80, 40)

for sample in sample_list:
        dataGen = DataGenerator([sample], preprocessor,
                                training=False, validation=False,
                                shuffle=False, iterations=None)
        ## Predict
        pred_list = []
        for batch in dataGen:
            payload = {"instances":batch.tolist()}
            pred_batch_ = requests.post("http://localhost:8501/v1/models/miscnn:predict", json=payload)
            pred_batch_ = pred_batch_.json()
            for key, value in pred_batch_.items():
                pred_batch = np.array(value)
            pred_list.append(pred_batch)
        pred_seg = np.concatenate(pred_list, axis=0)

        ## Postprocess prediction
        pred_seg = preprocessor.postprocessing(sample, pred_seg)

Hope it helps someone! Best, Constanza

[muellerdo]

Hey @covasquezv,

happy to hear that it worked out! Thanks for the update, added an information label so it can help other people.

Cheers, Dominik