search

google-research / bert

TensorFlow code and pre-trained models for BERT
https://arxiv.org/abs/1810.04805
Apache License 2.0
38.2k stars 9.61k forks source link

flask to deploy on line #263

Open mokundong opened 5 years ago

mokundong commented 5 years ago

when use flask deploy a fine tuned- classification task online, each post will reload the model. this takes about 4 seconds to respond, is there any solutions to avoid reloading?

aron3312 commented 5 years ago

94

You can refer to this. import predict_online function in run_classifier_predict_online.py. It will create a session and graph initializing at first, and you won't initial model every time you send a request

mokundong commented 5 years ago

94

You can refer to this. import predict_online function in run_classifier_predict_online.py. It will create a session and graph initializing at first, and you won't initial model every time you send a request

thanks!

hanxiao commented 5 years ago

@mokundong You may refer to bert-as-service for serving and deploying BERT models.

mokundong commented 5 years ago

@mokundong You may refer to bert-as-service for serving and deploying BERT models.

it works for classifier task?

anasuna commented 5 years ago

@mokundong You may refer to bert-as-service for serving and deploying BERT models.

it works for classifier task?

It didn't work for me for Question answering