search

awslabs / multi-model-server

Multi Model Server is a tool for serving neural net models for inference
Apache License 2.0
995 stars 231 forks source link

MXNet model service inference iteration possible bug #912

Open hassonofer opened 4 years ago

hassonofer commented 4 years ago

I suspect there is a mistake at (notice the inner for-loop): https://github.com/awslabs/multi-model-server/blob/70e8f1f622dcc0a6619125a8a23309ebf568ffbc/examples/model_service_template/mxnet_model_service.py#L143-L150

I suspect it should be:

        for d in model_input:
            if isinstance(d, list):
                for n in d:
                    if isinstance(n, mx.ndarray.ndarray.NDArray):
                        n.wait_to_read()
            elif isinstance(d, mx.ndarray.ndarray.NDArray):
                d.wait_to_read()
        return model_input

Otherwise we iterate over the same list inside the iteration.

Since MXNet Module might return list or list of list I think that was the intended purpose.