search

Nuix / Language-Translation-Integration

A script which integrates with third-party translation services like Google Cloud Translation or Microsoft Cognitive Services, providing a way to translate text of items in a Nuix case
Apache License 2.0
1 stars 1 forks source link

Timeout Issues Libretranslate-Docker #9

Open Rezo996 opened 7 months ago

Rezo996 commented 7 months ago

Hey there !

Me and my team have issues with the script producing quite a few timouts with the rubyscript running in NUIX 9.10, which accesses a Libretranslate (v. 1.4.0.) Docker container for translations. The issue we're facing is that although you can set a timeout when executing the LanguageTranslationIntegration script in NUIX, this timeout is not being passed on to the Docker container. For example, we set the timeout ( 120s ) and start translating, the first items were tagged "success" bc there is no processed text in Nuix. The next one, big Excelsheet, runs into timeout and should be skipped for beeing too large for the short timeout. But as a result, when the timeout expires, the script execution continues in NUIX, starting the next item, while the Libretranslate container continues to process the first translation request, causing the queue to grow. Eventually, the queue grows to about 94 elements, and then it stops accepting new jobs. How can I effectively pass the timeout to the Docker container?

The bulk of items are nearly 800 items, including some excels with over 1 million lines, about 2 MB - 10 MB big.

Anyone facing issues like us and may help?

Thanks !

Greetings, Rezo

JuicyDragon commented 7 months ago

Hello @Rezo996 I am not the author of anything really in this repository, but I have sort of become its maintainer.

I looked at the documentation for LibreTranslate on GitHub and neither the sections Settings / Flags or Configuration Parameters seem to specify a way to configure the timeout for LibreTranslate itself.

You might be able to use the configuration parameter --req-time-cost which they document as:

Considers a time cost (in seconds) for request limiting purposes. If a request takes 10 seconds and this value is set to 5, the request cost is either 2 or the actual request cost (whichever is greater).

I'm not sure I really understand what that setting is supposed to do.

There is also the parameter --frontend-timeout which defaults to 500 (500 / 60 = about 8.3 minutes). While the name suggests this is just the timeout for the built-in web frontend, I suppose it may also specify timeout for translations in general?