search

keras-team / keras-applications

Reference implementations of popular deep learning models.
Other
2k stars 910 forks source link

Instant predict in Keras #77

Closed PraveenAsh closed 5 years ago

PraveenAsh commented 5 years ago

I'm running into predict latency issues for deep models. A hybrid model built on top of VGG19 takes quite a while to predict. The application I'm working requires instant predictions. So is there a way where one could keep the weights loaded and allow the model to be in ready-to-predict state? Which otherwise would be reloading the entire model for every prediction or send a few samples to predict in an array at once.

Conceptually, the model should be listening to a request made ie be in ready-to-predict state, such that a sample passed creates a trigger which instantly makes predictions. Something like an Event-Listener in web-developer terms.

If already developed, please provide details or documentation for the same. Or even an alternative working solution would do.

taehoonlee commented 5 years ago

@PraveenAsh, It is hard for me to say that the event-listener idea is the scope of Keras. A Keras model already behaves keeping weights and predicting with the weights. Thus, you need to implement your event-listener because checking the data readiness depends on your environment and application.