search

thammegowda / nllb-serve

Meta's "No Language Left Behind" models served as web app and REST API
http://rtg.isi.edu/nllb/
149 stars 20 forks source link

Concurrent request error #19

Open Eric-chy opened 1 week ago

Eric-chy commented 1 week ago

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1463, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 872, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 870, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 855, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return] File "/app/nllb_serve/app.py", line 141, in translate inputs = tokenizer(sources, return_tensors="pt", padding=True) File "/usr/local/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 2882, in call self._switch_to_input_mode() File "/usr/local/lib/python3.8/site-packages/transformers/models/nllb/tokenization_nllb_fast.py", line 273, in _switch_to_input_mode return self.set_src_lang_special_tokens(self.src_lang) File "/usr/local/lib/python3.8/site-packages/transformers/models/nllb/tokenization_nllb_fast.py", line 295, in set_src_lang_special_tokens self._tokenizer.post_processor = processors.TemplateProcessing( RuntimeError: Already borrowed

Eric-chy commented 1 week ago

It works fine when I make a single request, but when I make concurrent requests, an error occurs.

sgowdaks commented 3 days ago

Hi @Eric-chy I am looking into this, could you please provide a sample code of how are you making concurrent requests?