Using state_is_tuple=False probably make the detection slow

[jialvarez]

Hi, I'm checking this project and Amazon Rekognition to extract text from spanish DNI images.

I've noticed this warning appearing many times:

WARNING:tensorflow:<tensorflow.python.ops.rnn_cell_impl.BasicLSTMCell object at 0x7f18cc56bc18>: Using a concatenated state is slower and will soon be deprecated. Use state_is_tuple=True. Detection is a bit slow even considering I'm using 8 CPU cores:

root@tataogg:/home/neonigma/UiPath_Document_OCR/inference_server# ./serve
Starting the inference server with 8 workers.
[2019-09-03 07:49:57 +0000] [4946] [INFO] Starting gunicorn 19.9.0
[2019-09-03 07:49:57 +0000] [4946] [INFO] Listening at: unix:/tmp/gunicorn_ocr.sock (4946)
[2019-09-03 07:49:57 +0000] [4946] [INFO] Using worker: gevent
[2019-09-03 07:49:57 +0000] [4950] [INFO] Booting worker with pid: 4950
[2019-09-03 07:49:57 +0000] [4951] [INFO] Booting worker with pid: 4951
[2019-09-03 07:49:57 +0000] [4952] [INFO] Booting worker with pid: 4952
[2019-09-03 07:49:57 +0000] [4960] [INFO] Booting worker with pid: 4960
[2019-09-03 07:49:57 +0000] [4968] [INFO] Booting worker with pid: 4968
[2019-09-03 07:49:58 +0000] [4983] [INFO] Booting worker with pid: 4983
[2019-09-03 07:49:58 +0000] [4984] [INFO] Booting worker with pid: 4984
[2019-09-03 07:49:58 +0000] [4985] [INFO] Booting worker with pid: 4985

root@tataogg:/home/neonigma/UiPath_Document_OCR/inference_server# echo $MODEL_SERVER_TIMEOUT $MODEL_SERVER_WORKERS $NGINX_CONF_PATH $TEXT_RECOG_MODEL_PATH
800 8 /home/neonigma/UiPath_Document_OCR/inference_server/nginx.conf /home/neonigma/UiPath_Document_OCR/inference_server/recog_model

I would like to speed up this recognition if possible:

INFO:Inference Server:Text recognition performed
INFO:Inference Server:Execution time: 57.39146566390991

I'm not very familiar with Tensorflow nor machine learning at this moment, but I can write code in Python. If I can submit any patch, please let me know.

Thanks!

[aarcosg]

Hi @jialvarez.

Sadly, this project is closed as we developed it for an UiPath competition the last year. However, I would like to give you some tips.

The inference task should run faster even with that warning. I think that it is taking more time because the models have not been loaded in memory (warmup). First, you should make a first call to the /ping endpoint once the inference server is up and running so that models are loaded in memory and get ready for the following endpoint calls. Warming up models is a very time-consuming task.

Let me know if your execution times are lower after that. I am sorry because we cannot provide to you further assistance.

Regards, Álvaro

[jialvarez]

Hi again,

I made a call to /ping endpoint and is taking 60sec too. I remember that in my first tests three days ago, the first call took this long time, but subsequent calls were acceptably fast (8 sec).

I can't get that working time again, because in every call is processing that warmup you're talking about. I mean, in every call is loading the models as if it was its first time. Is there any way to ask the software for storing the models in memory?

Thanks for your help!

[aarcosg]

Hi @jialvarez,

That makes sense. Probably you are running out of memory and Python's garbage collector is doing its job.

There is a singleton for each model and you can find all of them in the predictor.py file. I would recommend you to just warm-up the specific models that you will use. For instance, you can comment the line 198 so the driving license model is not loaded. Another possible solution would be making a call to a model's endpoint and ignore its running time once.

Note that all of the text detection models developed for this project use a lot of memory since they are based on deep neural networks and we aimed to achieve a high grade of accuracy rather than speed or low memory consumption.

I hope that these tips can help you. By the way, it was a project developed for a code competition. Currently, we have created a better OCR engine for Spanish ID cards that is being used by our clients. So this project is just a good starting point.

Regards, Álvaro