search

modrinth / code

The Modrinth monorepo containing all code which powers Modrinth!
https://modrinth.com
Other
982 stars 184 forks source link

Always manage own Java versions #663

Open brawaru opened 1 year ago

brawaru commented 1 year ago

Is your suggested enhancement related to a problem? Please describe.

The way Theseus currently manages its own Java versions is very inconsistent and very error prone. It only downloads its own JREs during the first launch, and only if it fails to discover already installed JREs, and will never download them again. This is hugely problematic as proven by several support requests in Discord:

Moreover, separation of Java 17 and 8 is confusing and not future-proof (what happens when Minecraft updates to, say, Java 20, do you just add another field? Isn't that too many fields?).

Describe the solution you'd like

I propose that Theseus manages its own JREs at all times unless user specifically overrides Java required to launch a specific Minecraft version. It will also only download (or update) JREs on demand when the version is launched using the managed JRE, not just download them all a single time during the first launch of Theseus.

Almost every Minecraft version manifest includes a Java version required to launch it, these versions are codenamed by Mojang:

Theseus seems to be already use this information to determine which version to use, so with the new solution it can use it to determine which managed JRE to download or update.

It also shouldn't be downloading singular versions, despite how Mojang locks them. Java is meant to be updated, therefore Theseus should keep them up-to-date, regardless whether Mojang locks them on specific versions (I doubt anything would break by switching from 17.0.1 to 17.0.3). Updates often bring security improvements and given how bad security in Minecraft modding scene, this won't hurt.

To do so, and to keep it clean, you will also need to use specific codenaming, which can be as simple as forming a string {kind}-{major}-{platform}-{arch} (e.g., jre-17-windows-x86_64 (this will allow to update existing major version, but keep it separate by platform and architecture, in case user wants to share the launcher config directory between multiple operating systems).

Maybe even have Java on versions on meta.modrinth.com and Daedalus to fetch the up-to-date versions to the bucket so that they can be partially download and deduplicated instead of fetching the whole archives from Azul. That is, do it the way Mojang does, except don't lock the majors.

This way you can avoid having to force a Theseus update when a new Minecraft version comes up that uses a yet unseen JRE, you will just have to add it to manifest on your servers and voila, all latest Theseus versions (that can fetch and read the manifest) work like nothing ever happened, support channel crisis averted.

Plus if you do hash based asset separation, you can deduplicate files on your servers and reduce amounts clients will need to download when versions update (just check if hash of existing file matches one wanted by the server, and if so, skip downloading, continue to next file), this way you don't have to download same LICENSE file several times, even though it doesn't change between the versions.

Only if user explicitly opts out by overriding the Java version in instance options should Theseus skip the JRE management and use whichever JVM user asked to be used.

Describe alternatives you've considered

Additional context

osfanbuff63 commented 1 year ago

Personally, I don't think Theseus/Modrinth App should force users into managing JREs for everyone.

Disk space

Imagine if you wanted to play Minecraft with some performance mods, but you had a working Java installation from, say, another launcher and you were running low on disk space. You wouldn't want Theseus/Modrinth App to download and install 5 different Java versions when you already had those installed elsewhere, since they would just take up space that you can't have taken up.

Java distributions

This one is a lot more for the technical people rather than your average user, but what if you don't want to use the Java distribution that Theseus/Modrinth App downloads, like using Adoptium over Azul Zulu or something like that?

In both of these cases, it would be tedious to have to change the Java path every time you make a new instance, so a solution could be to have an option in the universal settings to disable this behavior. Either way, I agree that the Java system needs some work, but I don't know that this is the solution.

Edit: Seeing #758, that would be another reason to not force managing Java versions. TL;DR if you don't want to go read that issue: Certain mods break when used with arm64 Java on M1/M2, so forcing you to use the "recommended" arm64 Java would prevent you from disabling that, even though you can't disable it as is.

brawaru commented 1 year ago

@osfanbuff63

  1. Your disk space concern is addressed by this line:

    It will also only download (or update) JREs on demand when the version is launched using the managed JRE, not just download them all a single time during the first launch of Theseus.

    However, you will need a manual intervention (to set the JRE you want to use) instead of Theseus automatically detecting and using whichever Java installation that you have. While this may seem annoying to you, there is very good reason to do it that way, because many non tech-savvy users don't have the most up-to-date or correct JRE installed, so blindly using them leads to all sorts of issues and hell to deal as CX (ask me how I know).

    By manually specifying JRE to use you acknowledge that you're leaving the zone of reliability provided to you by Theseus, and all your issues that happen with your custom JRE are entirely on you.

  2. Your distribution concern is also addressed:

    Only if user explicitly opts out by overriding the Java version in instance options should Theseus skip the JRE management and use whichever JVM user asked to be used.

    You would be able to use any JRE you want, but you will have to manually override it, and as said above, by that acknowledge that no guarantees will be provided to you anymore by Theseus.

I understand your concern about tediousness of specifying the JRE path for every instance, and yes, we perhaps could have an option in settings to override Theseus JREs, but it absolutely should not be the part of Onboarding as this would mislead users to believe that they have to specify custom JRE, no matter how you present it.

The reasons why I initially didn't want it to be a global setting is because there always can be an update to Minecraft which will use newer JRE (say Java 20), what do you do to handle this? Automatically show a field for each major? Ship an update to Theseus that will be rejected by some users because they're lazy to update?

Let's say we do it this way, but then JRE version selection may still fail you: maybe you want to run some version with Java 7 for whatever reason or some newer version, but global settings will are limited to the current majors (8, 17). Having no global setting, you'd know that you can simply change it in instance settings, and your setting would only affect that particular instance, so you're unlikely to cause a disaster by accidentally overriding the JRE globally. This may never happen to you, but are you really willing to bet this won't happen to anyone else?

Basically, the less paths there are to achieve something, the more straightforward and less prone to errors it is.

If you have any better ways to solve this problem, which absolutely accounts for the experiences of non tech-savvy users, then please write them down. But I believe Mojang solved it pretty well, and I haven't seen a single complaint about its system from my non-technical friends.

brawaru commented 7 months ago

Moreover, separation of Java 17 and 8 is confusing and not future-proof (what happens when Minecraft updates to, say, Java 20, do you just add another field? Isn't that too many fields?).

Guess what happened in 24w14a...

getchoo commented 7 months ago

this wouldn't be feasible on linux as binaries won't work across distros, especially less popular ones that use something like musl or don't follow the traditional FHS

i wouldn't implement this

brawaru commented 7 months ago

This sounds like a Linux problem, which Linux distributors will have to fix themselves. There can be a flag to building Theseus that will remove automatic JRE management and instead search for existing installation of Java otherwise throwing an error. I think the majority of Modrinth App users are using Windows and macOS, and for them this solution will work amazingly, and remove so much pain that the current approach is causing.

Added: I also think this will work on the majority of normal Linux distributions like Debian-based distributions. This sounds like some Nix and stuff problem, in which case ¯\_(ツ)_/¯. Flag should do it.

getchoo commented 7 months ago

This sounds like a Linux problem

assuming randomly pre compiled binaries will work anywhere on linux is a developer problem

There can be a flag to building Theseus that will remove automatic JRE management and instead search for existing installation of Java otherwise throwing an error

ok so the feature should still maintained internally...but only one on platform. odd, but ok

and for them this solution will work amazingly

i understand how automatically handling versions, installation, and updates does...but removing the most basic and common customization options for instances? if that's your goal, 100%! otherwise, this is just taking away features for some platforms for little actual reason.

this isn't an all or nothing problem. you can have automatic java management by default for a good ux, but still allow for these options to be overridden

I also think this will work on the majority of normal Linux distributions like Debian-based distributions

this isn't a guarantee, as dynamic library versions on the system will be updated. a binary may work on debian stable, but not testing. likewise, a binary may work on ubuntu 22.04 but not on 23.10

distributing dynamic binaries is not reliable on any distro, and we should know this already considering the deb is suffering from the exact issue described above

This sounds like some Nix and stuff problem

as said before, this isn't exclusive to linux distros like nixos, alpine, void, gentoo, etc. this is an issue with making broad assumptions about what a "linux" system has. when binaries are distributed correctly (like through flatpak or a repository package), the runtime environment is actually taken into account and prevents these issues; you're not just building for ubuntu lts and figuring every other distribution is similar enough

all of this isn't to say automatic java management can't or shouldn't ever be a default for linux, though. flatpak could handle this well, and the JREs might happen to be compatible with other distros. i just want to note it is very important for this to be configurable and probably off by default to be safe, so that packagers don't accidentally ship a broken experience (but we could still enable it here upstream). we're doing this at prism, for example

and lastly, i think if code is kept for configuring the JRE manually, it should probably be enabled for all platforms. most of the issues mentioned here could be resolved by improving java detection for game versions, avoiding the requirement of client updates for new version requirements by minecraft, and only using externally installed JREs when detected (the first 2 being required regardless of the options being kept). i don't see much point in removing a customization option that even the official launcher has support for

brawaru commented 7 months ago

This proposal does not suggest removing customisation options, and if you really need to, you should be able to override auto-managed JRE:

You would be able to use any JRE you want, but you will have to manually override it, … and yes, we perhaps could have an option in settings to override Theseus JREs. — Comment #3 by me

It proposes the removal of automatic fallback to JRE installed on the system and addition of lazy downloading of JREs whenever a version requiring a particular JRE is run.

As an example, here's the first time user experience on Windows. Our user has 32-bit Java 17 and 8 installed on their system for whatever reason. Here's their current experience:

  1. On first launch Theseus detects that Java 8 and 17 installed already and happily uses them without checking anything besides the versions.
  2. User creates a profile for 1.20.4.
  3. They try to run it.
  4. They receive an error because first 4 GB of their RAM are used already.
  5. They are confused, go to Modrinth Discord server and ask in #modrinth why the hell won't their game launch.
  6. They get 3 irrelevant suggestions and 2 scolds for not posting in #community-support.
  7. They make a post in #community-support.
  8. Get 5 more irrelevant suggestions and asked 2 times to post their Minecraft logs.
  9. Finally a Chad user lands in their thread and goes ‘um do you have 32 bit java installed? 🤓’. Turns out they are.
  10. < . . . >
  11. Finally, they installed normal Java 17 and ready to launch the game.
  12. Path invalid or non-functional.
  13. ‘Oh, just go to settings and click install recommended’ someone tells them. Which they do.
  14. Theseus downloads and installs Java 17, and applies the new path.
  15. They are finally able to run their game using auto-managed JRE.

Here's the new proposed experience:

  1. Theseus just launches.
  2. User creates a profile for 1.20.4.
  3. They try to run it.
  4. Theseus sees no JRE override for profile, goes to auto-detection mode.
  5. It sees that 1.20.4 requires java-runtime-gamma (17).
  6. It sees that there is no global JRE 17 override. Goes to auto-management mode.
  7. It sees that no auto-managed JRE 17 is installed. Thus it should be downloaded.
  8. It downloads up-to-date JRE 17.
  9. Minecraft is launched using auto-managed JRE 17.
  10. User enjoys the game even though their initial configuration was set to fail them.

On Linux distribution where auto-management is disabled the experience goes like this:

Path A — simple:

  1. <first 5 steps are the same>
  2. It sees that there is no global JRE override for this particular version either. An error shown that the user has not provided path for JRE 17 in settings.
  3. Game launch cancelled.

Path B — user-friendly:

  1. <first 5 steps are the same>
  2. It sees that there is no global JRE override for this particular version either. Goes to prompt-and-verify mode.
  3. User is prompted to provide path to JRE 17.
  4. They provide path to JRE 17.
  5. Theseus verifies that the provided JRE is indeed Java 17 and a 64-bit version. If not, a warning is shown and user is able to go past it.
  6. It saves a global override for JRE 17.
  7. Minecraft is launched using globally overridden JRE 17.
  8. Linux user secretly enjoys the proprietary product by disgustingly evil Micro$oft corporation.

Hope these examples clarify the intent of the proposal. This is by no means the final form of how the things would work, nor do these examples provide any solid details on how things work under the hood (e.g. effeciently downloading JREs), but the premise is there.

getchoo commented 7 months ago

As an example, here's the first time user experience on Windows. Our user has 32-bit Java 17 and 8 installed on their system for whatever reason

i feel this is a bad way to start out. this isn't representative of the common new user, who has probably only used the official client and maybe overwolf, with no reason to have these versions of java installed

They are confused, go to Modrinth Discord server and ask in #modrinth why the hell won't their game launch. They get 3 irrelevant suggestions and 2 scolds for not posting in #community-support. They make a post in #community-support. Get 5 more irrelevant suggestions and asked 2 times to post their Minecraft logs. Finally a Chad user lands in their thread and goes ‘um do you have 32 bit java installed? 🤓’. Turns out they are.

this sounds like a support issue. if the issue is this common, i would hope appropriate channels would have the resources with it - like a tag in a discord bot. the launcher itself could also detect this, not use it, and ease the problem

Finally, they installed normal Java 17 and ready to launch the game. Path invalid or non-functional.

i'm a bit confused here. if a user correctly installs a JRE (which is just running a setup.exe, something i'm sure most users are fully capable of), why wouldn't it work? is this assuming the launcher detected the path incorrectly? that termurin or similar don't work for most people?

‘Oh, just go to settings and click install recommended’ someone tells them. Which they do.

yeah this is a good solution for someone who doesn't have a working JRE installed. i'm not sure why anyone would download a third party one in the previous steps when this is built into the launcher

and as for the proposed experience

It sees that there is no global JRE 17 override. Goes to auto-management mode.

i don't understand why this has to be a specifically enabled override (or what i'm assuming to be, as you didn't just say "global JRE"). if a working JRE of the correct version is found on the system, there's no real issue in using it. the emphasis on here is working though, as obviously this is a poor idea otherwise -- prism implements checks for this as an example

as for linux

It sees that there is no global JRE override for this particular version either. An error shown that the user has not provided path for JRE 17 in settings.

this should be autodetected, like i said before (especially since it's super easy on linux)

It sees that there is no global JRE override for this particular version either. Goes to prompt-and-verify mode. User is prompted to provide path to JRE 17. They provide path to JRE 17.

ditto

Theseus verifies that the provided JRE is indeed Java 17 and a 64-bit version. If not, a warning is shown and user is able to go past it.

this should be done whenever a JRE is ran for the first time for all platforms. it would basically remove the issues user have with 32-bit installations, bad installs, etc., which would make the use of a system JRE ootb (if detected) pretty low risk

overall i think the improvements to automatically handling java here are great! i just don't see a reason as to why it's always a default regardless of what's on the system. JREs aren't all that small after all, and if a working + compatible version is already installed, you're really just wasting space with a new one. a lot of these ideas would also work just the same in this scenario, just replace the global override with what's on the system

i would also change the issues name, as it's a bit misleading and artificially limits the scope. i think a general java management improvement thread would be a lot more helpful, as we could really work on both the internal and external implementations