Build a separate, lightweight Minecolonies jar

[Eufranio]

It would be nice if you could build/provide a separate Minecolonies jar, without the heavy assets (images, sounds, etc), so we can reduce the jar size from 20MB to 2MB and happily upload to servers.

[Misunderstood-Wookiee]

You mean to provide an optional light-weight server JAR which does not have the assets that clients require? I doubt the devs are going to want to do this 20mb is already nothing really. If you cannot upload 20mb file to a server you have a bad host.

[Eufranio]

This ticket was just to remind them, looks like they have already this planned. Many people in this world have a bad internet connection, and with the update frequency of Minecolonies, i'd not be suprised if people started to be afraid of updating just because of the size.

[Misunderstood-Wookiee]

Each to their own I suppose, Professionally speaking from a fair span of time hosting various platforms over the years I would simply say to these people that they are not obligated to update to the very bleeding edge every time a new version is released, it is an automated Alpha build and there are many ways to automate code compile, if you thought of updating a 20mb mod every time the developer makes some tiny change e.g. (commits: tidy up code) its entirely your own fault for wasting time in doing so.

I would not bother updating a mod until a bug fix or feature release is available except under circumstances where code optimization takes place which improves backend performance. What I am really getting at is out here in the wild industry of computing most large consumer services won't update their software until mid-year or when it has been passed by the IT department for quality and reliability.

Obviously, this is different from Minecraft but some policies can be taken from the business world and applied too how you manage your servers for games. It really is up to the server admins if what version of something and when they update. Just because a new version comes out does NOT mean they have to update.

I remember plenty of times I would wait a week or so before rolling out any major updates which change how things work for users on the server or simply because the change was so minor it did not affect anything in a major way to warrant bothering shutting down the server and if I did shut down the server it would only be during routine maintenance days I would update anything at all. I dedicated one day a week for scheduled maintenance by the staff and updates, testing. If something would extend past a day I would give notice ahead of time, or postpone the update until I had everything tested and ready on a dev server without live users playing on it.

It's just silly to update every second day on a loaded server, it's quite annoying for the operator and its annoying for the end user. I still stand by the fact that 20mb is literally nothing. I am from Australia if you google our internet providers here you will find its well behind EU and the USA our average consumer upload link is around 6mb's if you are close to an exchange. (no that is not actual speed. actual speed is more like 1-2mb's up) I used to manage a server on less than that with an uplink of 1mb's or real speed of about 180kbs just fine and I would upload over 100mb when uploading new worlds. The trick to it all is giving yourself enough time that is why you schedule maintenance and not just update willy-nilly.

If they are working on a lighter version yea okay but for me personally keeping track of two versions one for the server and one for the client is a waste of my time as I have a modpack to maintain and a server why would I upload separate versions of the same mod when I can upload one version to both and be done with it, and let the server configs decide how much of the mod is available on the server and which parts I don't use. Take Recurrent Complex as an example for clients it has a GUI class unneeded for servers the config has an option to disable this non-essential feature on and run the mod in a light mode for servers if the client does not have the mod installed you will only have command access to the features and no overlays, the mod is not even required by the client so it's optional in the pack, hence I use Solder to maintain different versions of the pack a user version and a staff version one consumes more RAM than another but both work on the server with no missing server gameplay with the latter having all the non-essential things installed which end users need not bother with..

Really it just comes down to good management which unfortunately is few and far between with accessible hosting these days. Your server could be managed by a kid who literally brought a server and has no idea how to maintain it. I used to do backend support for people like that and let me tell you it's a brain cell killer.

If there are two version of this mod fine with me but I won't bother checking for a light version there is no need for this if your operators know what they are doing, new builds of Forge and things like that you really should be doing all of those revisional changes during maintenance and as I said already this mod is built automatically and this could be once every hour or once every time a commit is pushed to build server. Focus more on what the update brings or changes and if you think it's needed on your server then update during your scheduled timeframe. The time for mucking around testing new major changes or swapping mods is for local testing servers or server with staff only and will not affect the running of your live server you would then update the live server once you have everything large ready to go. I am not against multiple builds but this is exactly why Worldedit, for example, had an optional mod for the client it's not required to play its only required for ease of use for those who have permission to use the mod thus not everybody needs it.

[marchermans]

@Eufranio @Misunderstood-Wookiee Sadly this is going to be a near no go. The size of the jar is this big because of the sound and nbt files. Under normal operations these jars are signed. Making it near impossible to tamper with. Also it requires the same signature on both jars, server and client.

Which I can not strip the resources from one of them.

[marchermans]

On second thought I will keep this open to ensure better handling of it in the future.

[mkienenb]

@OrionDevelopment

Sadly this is going to be a near no go. The size of the jar is this big because of the sound and nbt files. Under normal operations these jars are signed. Making it near impossible to tamper with. Also it requires the same signature on both jars, server and client. Which I can not strip the resources from one of them.

It's unclear to me what the technical reasons are for why we cannot do this.

We've already shown that it's possible to move sound resources into a resource pack. We could distribute them as a resource pack or as a separate dependent jar. The same would be true for the other resources.

The hurdle I see is managing the logistics. We'd need to track the version of each jar we distribute instead of only a single jar and handle bug reports appropriately. We'd have to insure that end-users install all of the required jars in the correct places.

[Eufranio]

This wouldn't require a new version. It would be exactly the same jar, but without resources. The normal mod would still be usable on the server, and those who search a bit and find this server jar would be able to use it as well.

[mkienenb]

@Eufranio We will need to track the version on the main jar. We will also need to track the version on the jar containing the schematics resources. Otherwise it will be difficult to debug why something is not working that involves buildings.

In fact, we'd probably need to maintain 3 jars to do this best. One for the main jar, one for the schematics, and one for sound/images/etc.

[Raycoms]

The problem is this part:

"Under normal operations these jars are signed. Making it near impossible to tamper with. Also it requires the same signature on both jars, server and client."

So making two jars one with for the client and one without wouldn't be possible.

But distributing the sounds in an resourcepack which will be automatically downloaded or something similar is already on our list.

[mkienenb]

@Raycoms I see what you mean now. I wasn't considering keeping both a "heavy" jar and a "light" jar. I was thinking that there would only be a a light jar + schematics jar + sound/images jar. The server doesn't need the 3rd jar. Although since the server does need the schematics jar, the hassle of keep things separate is probably not worth it. It'd probably be better to have the light jar include the schematics.

[Raycoms]

The problem with this is that people need to download both. So, when we add this we would make it that it gets downloaded automatically on the client.

[Misunderstood-Wookiee]

I still don't understand the logic behind bothering with a light jar, all you are doing splitting up a mod into three parts like this is more work for the devs and trackers, more work for the end user and people will simply NOT bother installing the mod most likely due to the effort of self-tracking and updating three parts of a mod, modpack or not a modpack eventually you will have to update three parts for one mod and that at least for me and I am sure many others as well would be infuriating when it came down to tracking which part screwed up the server or client as a simple update to one part could glitch out all three and you would never know really because the schematics are just schematics no real code running there.

Look if three parts are what you suggested then I suggest sucking it up and at least pack the three jars like this.

Core Jar- Main mod (this provides the API and core functionality

Schematics Jar 
Sound/Image Jar

The two parts would get packed into Core Jar and unpacked at startup you would sign each at build time using gradle or whatever else to build the non-core parts into the core jar. Then leave it up too server owners to upload which parts to the server with the core part being mandatory.

Even in saying all of this you are still doing all this extra effort and work for a measly 20-30mb end user size which IMO is pathetic and a waste of time I'm sorry I just don't understand the logic behind creating more work for the dev team and end user over a few extra mb of file size. This is not a modpack this is not a multi-part zipped world as those can exceed well over 1gb this is a lousy 20-50mb if you can even play Minecraft without 2-3s block break/place delay you can upload 30mb and stop complaining.

I simply just don't agree and unless the devs can give me a solid benefit of this process I would be the first one here to complain about the format change because three parts are absurd and just creates drama's for everybody and there is no need for it. I already see people have trouble installing one jar for mods or an API and the mod people complain about that sometimes and struggle to imagine the struggle and time wasted if people had to explain there are three parts! and you need all of them client sided and then you have to think about how are you going to "self install" the other two parts if not done manually because Forge does not automatically go and collect your dependency mods and many people already use third-party clients nobody is going to drop say Technic Launcher for a homebrew launcher which is only used for installing this one mod.

The devs would have to do even more work and create an executable jar which downloads the other parts if you run the executable jar but this won't get it installed into the client because of prev reason people use custom launchers with mods and they could be located in any such folder convention and no single install location will work for everybody.

Edit: It's like a couple people just don't want to wait for a tiny file to upload and instead want to inconvenience everybody else, I get it, some people have shitty connections and this problem has existed since the dawn of ADSL/2 but game devs didn't go and remove content from an MMO simply because a few thousand users had a shit connection to the server.

If file size is so important to you two I suggest you just 7zip the damn thing and then send that to your server instead, almost ALL server hosts can unzip a zip file into a directory. If you have the host connection setup yourselves via a rented DEDICATED SERVER -

(NOT A RENTED GAME SERVER, I don't want you to be confused with that service)

then FileZilla has a server end version which can compress the transmission and turn a 20mb file and reduce the size during transmission down up to 45% almost half the original size and it works on large files too. This function only works if the server is running the FileZilla service and you use the FileZilla client to upload/download.

[mkienenb]

@Misunderstood-Wookiee We have to do this anyway to deal with translations.

After further discussion, we will probably keep schematics bundled with the main jar. There's not much benefit to doing otherwise right now.

https://github.com/Minecolonies/minecolonies/issues/2176

[Misunderstood-Wookiee]

@mkienenb Not really, you can just synthesise text to speech using Minecraft's built-in text to speech via your own synthesizer ad read from normal translation text file or provide an optional download and people can just replace the existing ones in the jar manually.

even if you split the mod up how would you get around the problem of three jars needed to be manually installed and they would be dependent on each other, I saw a mention of AUTO install and download but how? you cannot execute a jar on a secure uploader (host) and you cannot simply point to .minecraft because different launchers will install mods differently and manage Minecraft differently, for example, Technic Launcher will have a folder named after the modpack itself and provide completely instanced installations and the Minecraft version is independent of whatever the modpack is as the Minecraft jar is not packed with the mods you can cycle versions on any pack anytime.

Many packs even use solder which means you update or downgrade a pack. The only way to really do this is to pack every part into the main jar and have those extract when the client or server starts.

Just like how some other mods will extract its core lib at startup on the first run, you would pretty much build them at the same time and gradel or your build server would compile them then a final build script would pack those into the main jar thus preserve the signature signage and then leave uploading whichever parts to the server to the operator by extracting the parts they need or letting the client do it after the first run...

Three jars manually isn't an option I would not even support multiple language audios if that was the only solution I would pack the translations into a zip and throw that onto GitHub with manual install instructions or self-run bash or bat file next to the jar file. I have never seen a three-part JAR based mod before which did not self-extract the other parts. I would understand if additional split was optional but they are not as the schematics are required on server and client the core is req the sounds/images are the only things NOT required on the server, and as it stands now the whole thing is under 45mb the extra effort to do this is greater than the reward of doing this, I can already see what is going to happen. People are going to start complaining and getting confused on the format change and people will start sounding like broken record players repeating the same advice over and over again to people who don't get how to install it or how to get it on the server correctly.

I am just trying to say that in doing this experience has shown that more complexity with your end product = more support time needed and more repeating one's self, the goal for any project should be - to keep it simple stupid, otherwise known as the [kiss] policy the backend can be a mess of things but the end result has to be clean and simple.

[Misunderstood-Wookiee]

Even @OrionDevelopment Thinks the providing the resources on GitHub as individual components are the better option, absolutely never should this turn into a three-part install procedure for getting just the English version running. I am not telling anyone what to do in-case it comes across that way I am just saying its going to be a nightmare for everybody else who's not bitching about 30mb.

Some of us have modpacks with over 100 mods to maintain and others have less but most of us on average have at least 30. The only people who got it easy with server hosting are Spigot users who can throw everything on the server as little 1-15mb plugins and be done with it. If people are Forge modding with shitty connections that is entirely their problem IMO. Nothing is even 30mb these days most simple patches for a game are over 2Gb nowadays, it's about time everybody in the world finally accepted this fact and that nobody uses dial-up anymore so file size is not really a huge concern anybody able to play with a decent latency can download or upload files up to at least 100mb without any real struggle especially if you are a web dev.

If it really comes to shove then at least maintain the full kit as it is now and a lighter multi-part for the whinging people all that would really take is an extra build script.

I just to point out that I also said if they are hosting via any reputable hosting service they can just ZIP the JAR up first which will probably net them a ratio of around 75-80% compression and make the file size somewhat near the 8-12mb region depending on compression extension used number of words.

[marchermans]

Since this discussion seems the be getting out of Hand. And I personally think it should not matter for a Server admin if he has to download 20MB or 2MB (He still has to run the command) I am going to Close and lock this discussion. If you REALLY NEED THIS, have a VALID AND GOOD usecase for it (so not, because it is easier for Person xyz) and give me a notch in the right direction when it comes to a solution, contact me on discord and I will be happy to add it to the buildsystem.

But for now, since nobody seems to be able to hit the nail on the head (so to speak), when it comes as to what is required and why. This discussion is going to end.