search

zaidalyafeai / ml-projects

Implementation of web friendly ML models using TensorFlow.js. pix2pix, face segmentation, fast style transfer and many more ...
MIT License
647 stars 254 forks source link

How to get faster prediction time #5

Open Aimantech opened 5 years ago

Aimantech commented 5 years ago

We are working on license plate recognition.We used a model converted from keras h5. But it is taking long to get loaded. The time for prediction is also high.

zaidalyafeai commented 5 years ago

@Aimantech, what is the architecture of the model and its size ? Are you converting a well know model or you are training form scratch ?

Aimantech commented 5 years ago

Training from scratch. We are following this keras example https://github.com/keras-team/keras/blob/master/examples/image_ocr.py. Size of the model is 17MB.

Aimantech commented 5 years ago

Model summary: capture

zaidalyafeai commented 5 years ago

@Aimantech , you said the loading time is long but compared to what? Are you loading the model from a server or local machine? If you are loading from a server then the speed of your internet connection is a factor. On the other hand, the prediction time should be fast as the model is not that huge.

Aimantech commented 5 years ago

Model is loading from github repo. Other tensorflow.js examples, including your's, works well. So internet speed may not be the problem. This warning is thrown when the model is being loaded:

Orthogonal initializer is being called on a matrix with more than 2000 (786432) elements: Slowness may result.

zaidalyafeai commented 5 years ago

@Aimantech, can you check with less parameters for the GRU layers. There might be a bug there. You don't have to retrain the model. Just convert an empty model.

Aimantech commented 5 years ago

You meant pruning?

Aimantech commented 5 years ago

`

@Aimantech , you said the loading time is long but compared to what? Are you loading the model from a server or local machine? If you are loading from a server then the speed of your internet connection is a factor. On the other hand, the prediction time should be fast as the model is not that huge.

The same model, when run with python keras takes about 0.2s. But in tensorflow.js the time for prediction is about 1.5min. In both case the backend was cpu.

zaidalyafeai commented 5 years ago

@Aimantech, not sure about the prediction time when the backend is cpu. I have not experimented with that. Maybe you could repost the question in https://groups.google.com/a/tensorflow.org/forum/#!forum/tfjs to get a precise answer from the developers of tfjs.