search

google-research / bert

TensorFlow code and pre-trained models for BERT
https://arxiv.org/abs/1810.04805
Apache License 2.0
37.47k stars 9.53k forks source link

Squad_prediction #522

Open Radeeswar opened 5 years ago

Radeeswar commented 5 years ago

How to use bert-squad model to predict fastly or how to load once and predict multiple inputs.(Since it takes 25-30 sec for one single input)

satishp962 commented 5 years ago

Yes, even I am facing the same issue.

iris-qq commented 5 years ago

has anybody solved this problem? pls,help me ,3q

Radeeswar commented 5 years ago

Try exporting the model. We get saved_model and variables. Use it for predictions. Check the below issue to export or even tensorflow . https://github.com/google-research/bert/issues/146

SherL0cked commented 4 years ago

@Radeeswar Hi, i'm wondering have you solved the performance issue? seems like post-process takes most of the time(predict only takes less than 1s), the iteration over whole features and examples cost too much. Please let me know if there's anyway we can bypass... Appreciate!

Radeeswar commented 4 years ago

@Radeeswar Hi, i'm wondering have you solved the performance issue? seems like post-process takes most of the time(predict only takes less than 1s), the iteration over whole features and examples cost too much. Please let me know if there's anyway we can bypass... Appreciate!

Hey @SherL0cked , Firstly the performance is better not too good. And the thing is once we export saved model then try to remove unnecessary steps in post processing and I used the below to load saved model. ---> tensorflow.contrib import predictor ----> predictor.from_saved_model(model) Example of removing steps like: Squad predicts 20 outputs for a input so reducing helps based on requrements. Ty!

SherL0cked commented 4 years ago

@Radeeswar ! That's pretty what I'm doing right now, as I'm working on a chatbot to answer the question in a long context, so I manually merged examples.doc_tokens from read_squad_examples and increased seq length, it works good so far. Thanks again for answering!

luke4u commented 4 years ago

@SherL0cked, any chance you can share insight on token length limit(512). If the input is much longer, how to process? I am thinking to break down input, but probably not sensible, because each chunk produces an answer, and correct answer can have low score than wrong answer.

yijinsheng commented 3 years ago

I successfully save the model follow this ,but when I try to predict the answer , I got the error ValueError: Cannot feed value of shape (1,) for Tensor 'input_example_tensor:0', which has shape '(8,)' Anyone can help