Closed KineticCookie closed 6 years ago
zhreshold
commented
6 years ago You need to provide the data shape if you are trying to run infer_shape. Docs: https://mxnet.incubator.apache.org/api/python/symbol/symbol.html?highlight=infer_shape#mxnet.symbol.Symbol.infer_shape
KineticCookie
commented
6 years ago @zhreshold thanks for pointing it out.
But the thing is, what if I don't know what data shape is for current model?
In this tutorial shape is basically hardcoded: mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))], label_shapes=mod._label_shapes)
I try to abstract over that and can't find a proper solution.
KineticCookie
commented
6 years ago The broader question: Is there a way to get information (names, data types, shapes) about model inputs and outputs using only model persistence mechanism? (like Tensorflow model signatures)
zhreshold
commented
6 years ago You can't get any output shape without input shape, that's true for every single library. If you would like have attributes of layers, gluon do have it.
from mxnet import gluon
net = gluon.model_zoo.vision.get_model('resnet152_v2')
print(net)I executed your snippet, and I can't find information about shapes model was trained with.
ResNetV2(
(features): HybridSequential( /* layer attributes */)
(output): Dense(2048 -> 1000, linear)
)Layer attribute doesn't contain information to get a model input and output shape. Moreover I can't find a data types for each layer.
I made an additional search and found this issue #7641 with @jeremiedb explaining shapes implementation:
There's typically no need to specify shape of data input when building the symbolic network. This will typically will be set at training time when the model is bind and the shapes infered from what the iterator provides as input data. This allows the same network to be trained with different batch sizes.
Seems like mxnet infers information about shapes and data types at training, but doesn't store it in model files. infer_[shape, type] methods are the only way to get this info, but they require hardcoded variables.
I try to implement Tensorflow Serving-like server that could handle any exported mxnet model, and serve it via HTTP api. But to do this I need to know:
If I put away my serving case: I got a mxnet model from datascientist to use in my app. Model ships with no documentation. I can't contact datascientist either. Is there any way to use this mxnet model, considering I have no clue what data was used to train this model?
zhreshold
commented
6 years ago You might want to checkout mxnet model serving @kevinthesun
kevinthesun
commented
6 years ago @KineticCookie If you don't know exactly what a model does and what kind of inputs it accepts, it might not be a good idea to directly use it in your app. For model inputs, there can be many variables you need to know to use the model for inference. For example, what types of inputs does this model expect, image or text or combination? What is the input data shapes? Is it a fixed shape or can be a variable?
A possible way to manage your own model zoo for serving is to have some signature files to record all the necessary information to use this model. However, you need to know the input information in the first place.
KineticCookie
commented
6 years ago @kevinthesun
If you don't know exactly what a model does and what kind of inputs it accepts, it might not be a good idea to directly use it in your app.
I understand your concerns, but the sole purpose of my app is to provide a simple way to expose models as web service. If I train model with dataset of int32 and 300x300 shape, I think that is obvious that I intend to use it with the same shape and datatype. The point is, why mxnet doesn't provide information about datatypes and shapes, which are already inferred and known in the training process, when I export model?
For instance, in TensorFlow:
When I import a model, I know about model inputs/outputs, with their types and shapes at a runtime. This helps me to validate data and pass it to the model. This also gives me an opportunity to create a documentation or an interface, so external users are able to use this model.
In conclusion, mxnet doesn't have a Signature mechanism and it infers shape and data at the training. So, why doesn't it save this information along with model?
kevinthesun
commented
6 years ago Take a look at https://github.com/awslabs/mxnet-model-server. To serve and manage models, you may need many information, such as input data shape and input data type(It can be both numerical type or MIME type). You may also want to record the model version and dataset used for training. I think here what you want is something similar to model signature file in mxnet model server. The design is to decouple all these model related information from mxnet model saving system. With save_checkpoint function in mxnet, you just save model graph definition and parameters. Then in serving framework, you provide all the information you need and save them as signature file. In this way you can store and manage them for serving. If all these information are added into mxnet save_checkpoint, it may be too heavyweight for mxnet standard model file.
Currently this framework is not in mxnet yet. But you can use the similar way of managing your model for web service, it can be just a signature json file. In the future, mxnet will add serving framework, similar to tensorflow serving. At that time, similar function to create signature file will be introduced in that serving framework.
KineticCookie
commented
6 years ago @kevinthesun thanks for the detailed explanation. 👍
Description
Hello. I ran to a problem with a model described in https://mxnet.incubator.apache.org/tutorials/python/predict_image.html I tried to infer shapes for model inputs but encountered an error.
Environment info (Required)
Package used (Python/R/Scala/Julia): I'm using Python 3.6 package.
Error Message:
infer_shape error. Arguments:
MXNetError Traceback (most recent call last)