NyashniyVladya / Translator3000

Automatic translator of games made on Ren'Py engine.
https://nyashniyvladya.github.io/
GNU General Public License v3.0
124 stars 18 forks source link

avoid in-game delays #58

Closed wangling12 closed 2 years ago

wangling12 commented 2 years ago

Already waiting _database_lock in add_translate_to_local_database, and waiting _hdd_lock and database_lock in backup_database. Therefore, there is no need to lock here, especially when there are HTTP requests that may consume a lot of time.

NyashniyVladya commented 2 years ago

This method calls the dictionary self._database, which can be read at that moment in another thread that translates strings with the same function. In your fix, there may be repeated addition of the same string to the database and repeated translation of the same string through the translation service. The backup and add translation methods are standalone and are not only called from this place, so a lock is also needed there.

I definitely reject the merge.

wangling12 commented 2 years ago

Anyway, this is an important point to be optimized.

The following is the flame diagram I obtained before and after the modification. In order to maintain fairness, prescan is turned on: out out1

We can see that the 118 line translate function takes more time to wait for the lock than the other functions of the game combined. And this situation will further deteriorate with the fluctuation of the translation server network.

NyashniyVladya commented 2 years ago

Imagine the following situation: the main translator thread requests the translation of string X, while the prescan thread also requests string X. Without a lock, they will both translate this string twice and write it twice. At the same time, if at this point the user requests, for example, a manual backup of the database, there will also be asynchronous access to the dictionary, without a lock, which is unpythonic-style. Not to mention the fact that the optimization will benefit only in one case: the simultaneous operation of the translation thread and prescan, which is in itself not recommended, because services will ban for requests without a timeout.