Translations stop functioning after a period of time

[BluSpring]

After a certain period of time (normally around 12 to 24 hours), the translations will suddenly stop working. This is caused by a Java standard library problem, and needs to be worked around.

For now, if translations suddenly pause, just run /unitytranslate debugreload. You may need to run this multiple times.

Technical explanation

Apparently, Java just leaves some HTTP connections open even if they have failed, and as a result they just don't end up closing/releasing, which just straight up causes a deadlock if there are too many HTTP connections. The reason why /unitytranslate debugreload fixes this is because I recently moved translation API calls to use a ForkJoinPool, and the reload would reset the ForkJoinPool and forcibly close all HTTP connections in that thread pool (which also explains the log spam if you have -Dunitytranslate.printHttpErrors=true enabled).

Further explanation towards this problem can be found in this Medium post.

Below is some information I stored previously just to explain the problem, if you want to read more about what I considered, here you go.

Discarded Solutions

Using Ktor for HTTP requests

Due to an actual problem with Kotlin on Forge 1.20.1 and 1.20.4 (thedarkcolour/KotlinForForge#86), Ktor is completely out of the question from being used as a solution. I was forced to throw Ktor out the window (e842f607925d42d91042c24a808f8c0c1b8b277e) just to be able to support Forge, and Connector couldn't seem to handle Ktor properly either.

Using OkHttp for HTTP requests

Just like the prior problem with Ktor, OkHttp is not a viable solution due to its Kotlin usage.

Rewriting the HTTP handlers to use the logic provided in the Medium post

This ends up becoming way too much effort for what it's worth, especially because this is an incredibly slow problem to fix (requires multiple concurrent translations, appears to only occur after a period of 12-24 hours or so, etc.), so this is not a viable solution.

Possible Solutions

Rewrite HTTP handlers to use Apache's HTTP Clients API

This is what 1ea2dd8a7dd98135799863c7c7ff898b4ea30011 does, but we need further testing to see if this actually helps.

[p0t4t0sandwich]

Reading your post there, most of those issues can be solved with shading+relocating, which doesn't have it's own myriad of problems, and works fine in modded environments, even in cross-plat or mono-jar environments. (*when used properly obviously) I don't know if you're speaking from personal experience or it was something you've heard somewhere, but the gradle shadow plugin works completely fine with most, if not all modding buildtools (even when taking reobfuscation into account). Shading+relocating also has the added benefit of being able to be minified, wheras JarInJar'd dependencies cannot, further reducing your Jar's size.

If you need examples, I have plenty in this repo here: https://github.com/p0t4t0sandwich/TaterLib

[BluSpring]

Speaking mainly from personal experience, seeing as I've done it many times before. it's caused a lot of problems for me whenever JiJ was not an option, not particularly the fault of Shadow itself, rather the libraries I usually needed to bundle.

In some few cases, a library would rely VERY HEAVILY on having its package name unchanged (one example is LWJGL, which is why server CUDA support isn't available in UT), which then requires either untangling what in the world is going on (which isn't really feasible), or just throwing it right out the window.

btw, to clarify, the reason behind LWJGL's requirement last time the library needed to be shaded (unrelated to UT) was because of an ImGui library depending on LWJGL, more specifically a very specific version of it that wouldn't work if you excluded it. The Axiom mod previously also had the problem of needing to shade LWJGL, I'm unsure if they still do that now though.

Additionally, with shading, sometimes it would also include some other libraries that I would have to determine if I actually need them and if so then relocate them manually, and if not just throw them out the window, which then repeats the cycle of "will it crash if I relocate it?" and stuff like that. It's not fun.

And unfortunately, the problem with Ktor and OkHttp being unable to be used in Forge environments still wouldn't be solved with shading either way, there's still some other issue relating to classloading on Forge because of that god forsaken module system.

So overall Apache's my best bet, and while I would definitely try to shade-relocate it when I have the time, I would really just. prefer if I didn't have to.

[p0t4t0sandwich]

Fair enough yeah, came across this and thought I'd throw in my two sense. Never come across that package-rename issue specifically, but that sounds like an absolute compatibility nightmare.

Just checked, and I've bundled OkHttp into a project here, and it's worked fine in Forge with no module collision issues. I'd assume because I've bundled and relocated the Kotlin dep as well, which given your env would be it's own hassle to try and relocate without buggering up your own Kotlin deps (you'd need to create a nightmare shadow config and exclude any of your project's paths since you use Kotlin). Further increasing that trial-and-error-relocation phase that you're wanting to avoid. So yeah, sounds like you're stuck between horrible gradle hell and a hard place lol. shadowJar#minimize() does help a bit to limit the amount of do-I-need-this-transient-deps, but still leaves a decent amount left over to sift through. Though if you ever do end up trial-and-erroring some gradle, I'd be down to bounce some ideas around.

(As an aside, I noticed your work on Kilt. I haven't had the chance to properly test it, but I've added some preliminary support to my cursed cross-plat lib, nifty work all around!)

[BluSpring]

I have rewritten the HTTP handler to use OkHttp now (112706c7e5c9852ed7199080724e94718d374f43), and I'm hoping to god that this solves the problem once and for all.