search

neo-ai / neo-ai-dlr

Neo-AI-DLR is a common runtime for machine learning models compiled by AWS SageMaker Neo, TVM, or TreeLite.
Apache License 2.0
492 stars 106 forks source link

Model inference results on Inf1 inconsistent with expected results #175

Open hasanp87 opened 4 years ago

hasanp87 commented 4 years ago

I was able to compile , deploy and run inference on Inf1 for a model created with mxnet 1.6.0 . I am using a pre-trained gluon cv i3d rennet model https://gluon-cv.mxnet.io/api/model_zoo.html#gluoncv.model_zoo.i3d_resnet50_v1_kinetics400. The inference results does find the right predicted class , however the probability value is only 20.1 %. If i run this model standalone from here https://gluon-cv.mxnet.io/build/examples_action_recognition/demo_i3d_kinetics400.html , it gives the same predicted class but with a probability of 99.9 %. So, clearly neo compile is not getting it right here ? how can we explain this, thanks

saranyailla commented 4 years ago

I have a similar issue with resnet50_v1 on x86_64 machines(ubuntu and deeplens). It seems to predict correctly on raspberry pi.

kkwan-sme commented 3 years ago

@hasanp87 @saranyailla I know your posts are now ancient history and there's no excuse for not tracking the posts and responding in a timely manner. Still, I wanted to see if you managed to resolve your issues. If you experience any issues with the current Neo compiler, you can send us a description of the issue and the compilation ARN to amazon-neo-feedback@amazon.com and we can follow-up with the team.