bbepis / XUnity.AutoTranslator

MIT License
1.93k stars 289 forks source link

Inconsistent/Temporary Translation #5

Closed AoiRice closed 5 years ago

AoiRice commented 6 years ago

Hello, for some strange reason this plugin only temporarily translates my game. For example, in Koikatsu I'll enter the character maker and everything will be translated, and as soon as I exit the maker the auto translation ceases. Another example of this issue is when I enter the class selector, most of the Japanese text will have been translated, and when I've started the new game or if I back out to the main menu, the translation ceases and every piece of text is back to Japanese. I've used this plugin before and I've never had an issue with it until now, after reinstalling Koikatsu. I appreciate any help I receive, thanks.

TokcDK commented 6 years ago

Yesterday I also found utilities wich is using Gautotranslation on translate.googleapis.com stopped working from time to time. After this I found G server started to return 503 error after small time of using translation function. Then I checked request on translate.googleapis.com in browser and g server said about detected suspicious traffic and temporary block. Then I attached proxy to one of using gautotrans apps and g also was blocking them quiet faster after 1-5 minutes with same 503 error. It allow translation only for browsers. I suppose yesterday Google started to block requests from 3rd party apps after some time. I checked several apps with same result. As I understood even spam prevention of this plugin like translation cache is not helping from blocking. It is bad of course for all who is using apps to understand text in japan games like this plugin or any utils like Chiitrans.

TokcDK commented 6 years ago

To bbepis. I didnt test it enough to talk about it works but try to see here about what was changed in fresh sources of utility in target HF theme to resolve the problem and make same fix for the XUnity.AutoTranslator. .

AoiRice commented 6 years ago

Interesting insights, it's great that you've got on it and investigated so quickly. I feel that it'd be a more easy solution to just add a more accurate translation site for optimization to substitute for Google Translate rather than trying to bypass Google's third party access blocking.

notanumber6 commented 6 years ago

Is there a more accurate translation site, though?

ghost commented 6 years ago

I think this works:

https://github.com/Scrublord1336/XUnity.AutoTranslator/releases

Himatori commented 6 years ago

Works for custom maid 2 and custom order maid 2?If it works, how to install it?

AoiRice commented 6 years ago

Tried the fix @Scrublord1336 implemented and I can confirm it works, just like how the AutoTranslator was. Hoping @bbepis takes notice of this.

ghost commented 6 years ago

I have been looking a bit further into Sinflower's fix for Translation Aggregator in order to actually understand it.

It seems that he is using two variables, something referred to as TKK. These two values are hardcoded in 2.6.0 (and Translation Aggregator), but is actually something that can be extracted directly from the gtrans web UI source.

In my experience these two values often change, which probably means they will be invalidated at some point in the future and 2.6.0 will break.

Sinflower commented 6 years ago

It seems that he is using two variables, something referred to as TKK. These two values are hardcoded in 2.6.0 (and Translation Aggregator), but is actually something that can be extracted directly from the gtrans web UI source.

In my experience these two values often change, which probably means they will be invalidated at some point in the future and 2.6.0 will break.

While it is true that the TKK values change (I think they change every day or even every fresh access to the website) that does in fact not tend to break the calculation, I had values from over two years ago still producing valid results. Which brings me to

Obviously the fix is probably a bit fragile given that it just takes them changing the way they calculate the checksum for this to break.

While here as well it is also true that a change will most likely break any system using this calculation, this calculation has been consistent for over two years (maybe longer). Fun fact, baidu uses the same calculation to produce their checksum.

In my experience, this is an acceptable level of consistency considering that this is neither an official API nor actually intended.

In case you would like to build a more elaborate system e.g. load and execute the original javascript functions, you can find them here: https://translate.google.com/translate/releases/twsfe_w_20180723_RC00/r/js/desktop_module_main.js Just search for one of the large magic numbers in the calculation. As a note however, google tends to update this file and while they do not change the calculation, they change the function names.

Sinflower commented 6 years ago

It did exist, however it was not used because up until recently it worked without the checksum. Additionally google now requires the cookie provided by translate.google.com to be present or else it will stop after ~50 translations.

Sinflower commented 6 years ago

I just looked at your current implementation of the checksum and noticed you used the first release I posed which contained a bug in the checksum calculation. I made a mistake while converting a part of the Javascript code to C++, to be precise, there was a problem with the cases inside the loop. You can find the (hopefully) correct loop code below.

for(size_t v = 0; v < r.length(); v++)
{
    int64_t A = (int32_t)r.at(v);
    if(128 > A)
        S.push_back(A);
    else
    {
        if(2048 > A)
            S.push_back(A >> 6 | 192);
        else if(55296 == (64512 & A) && v + 1 < r.length() && 56320 == (64512 & r.at(v + 1)))
        {
            A = 65536 + ((1023 & A) << 10) + (1023 & r.at(++v));
            S.push_back(A >> 18 | 240);
            S.push_back(A >> 12 & 63 | 128);
        }
        else
        {
            S.push_back(A >> 12 | 224);
            S.push_back(A >> 6 & 63 | 128);
        }

        S.push_back(63 & A | 128);
    }
}
Sinflower commented 6 years ago

Yeah if you do not use persistent HTTP connections you should not face the cookie problem, TA uses a persistent connection (used by all web based translators) which causes the problems.

As for the code, yeah that indeed looks correct, maybe I looked at an old commit.

Sinflower commented 6 years ago

I ran some stress tests on my solution (between 5 and 10 request per second I think) for around 30 min. without any problems.

Sinflower commented 6 years ago

During the first request and afterwards every 100 requests I update the "requestHeaders" variable by adding the cookies retrieved from the website, during the actual request with "WinHttpSendRequest" "requestHeaders" is used to provide all headers including the cookies.

Sinflower commented 6 years ago

I get the cookies from: https://translate.google.com/

During GetCookies I simply download said page and retrieve the cookies.

phman112293 commented 6 years ago

again stopped working, after a certain number of words stops translating

phman112293 commented 6 years ago

Thank you very much for your help, for the moment it works correctly with the version XUnity.AutoTranslator-BepIn-2.8.0 has exceeded the number of words where I stayed, anyway I inform you if an irregularity occurs. And thanks for allowing me to be the first to try the new version

Again, thank you very much for your work and your effort

tureg commented 6 years ago

So what does a user with no programming knowledge (at all) do to fix this, with CM3D2? Also, I can only do Sybaris for some reason, I did the Sybaris 2 to BepInEx migration pack and used @Scrublord1336 's unreleased autotranslator above but the game crashes before launching with Bepinex for some reason

Thanks in advance

tureg commented 6 years ago

@Scrublord1336 that would be amazing. Looking forward to it, thank you!!

ghost commented 6 years ago

So, the latest version does have a UnityInjector installer as well.

Right here.