Create an additional shaded API jar

[kashike]

It would be nice to have an API JAR with dependencies shaded for developers who do not wish to use a build system.

[Mumfrey]

Wouldn't that encourage people to not use a build system though?

[ElgarL]

Some of us old-timers do not like build systems ie maven/gradle

[Mumfrey]

I'm an "old timer" so to speak, and still use ant and ivy for things in my own projects, just because you don't want to use maven or gradle doesn't mean you should not use a build system. Why would you not use a build system, it makes no sense?

[Zidane]

While @progwml6 will argue against this wholeheartedly (and I'm sure many others will), I see the points being made.

Build systems are wonderful and I'd never recommend anyone to NOT use one but its just that, a recommendation. It shouldn't be a requirement for SpongeAPI to simply HAVE to have a build system (or go out and fetch the libs yourself).

While the shaded API jar won't be the norm nor be used in the impl, there is nothing preventing us from providing a -all jar for the API.

It comes down to choice, should we punish people who use their freedom in software choice by choosing to NOT use a build system by having to seek out all the libs themselves?

[Mumfrey]

It's not punishing people to simply observe best practices. If people want to deviate from best practice that's entirely their own right, but a platform should encourage good practices and not using any form of build system doesn't fit in that space.

[ElgarL]

Oddly I HATE 'best practices' and 'norms of programming'. Learn them of course but the greatest inventions and leaps forward were made by people not trained in the field. People who were not constrained by the limitations of being taught what is right and what is wrong.

Learn how to do things correctly, then go out and break everything by doing it your way.

[progwml6]

Use a build system. We agreed from the beginning of the project that there will not be a shaded API jar

[Zidane]

@Mumfrey

We do encourage and have done so this entire time. You'll not see one ounce of non-build system usage throughout our docs and elsewhere.

Again, besides principle (which one could argue) is there simply any reason why there couldn't be a shaded API jar provided for download?

[Zidane]

@progwml6

When an issue is being discussed, that doesn't mean you can close it.

[Mumfrey]

Oddly I HATE 'best practices' and 'norms of programming'. Learn them of course but the greatest inventions and leaps forward were made by people not trained in the field. People who were not constrained by the limitations of being taught what is right and what is wrong.

Learn how to do things correctly, then go out and break everything by doing it your way.

That's a great ethos and I respect it, but it still doesn't really explain why you would not use the tools provided. If you're not using a build system and doing everything by hand, then it's not fair to ask for special provision be made the existence which could mislead others into believing it's a good idea.

I mean, I can understand you not liking maven or gradle, I don't either. But using (as a minimum) ant in your project just saves you from doing the entire build process by hand every time, and will do things like fetch all the dependencies for you.

[ElgarL]

Mumfrey I do use Ant but I believe an API should provide all of its dependencies to the end user (programmer). It should never be returning unknown objects to an IDE without also providing the corresponding class. An API should be complete in that it can be added as a dependency and fully used without any further steps required by the programmer. Depending on a build system breaks specific features in IDE's like jar exporting.

Build systems are great, but should never be required.

[Mumfrey]

If you're using ant then why aren't you fetching dependencies with ivy? It'll pull everyhing you need down in one swoop and you have full control of where those jars go (ivy "retrieve" can be configured to just dump everything in a folder called "libs" for example (which is what I do)). Then you're free to do what you want with it.

An API should be complete in that it can be added as a dependency and fully used without any further steps required by the programmer.

I'm not really sure this is entirely true, many libraries are distributed without their dependencies, but given those dependencies are supplied in a handy bundle (the ivy.xml or maven-metadata.xml) they can be retrieved easily

Depending on a build system breaks specific features in IDE's like jar exporting.

As I've said, I support you in the whole "build systems are overrated" thing, but I don't understand why this means managing dependencies by hand. There are tools available for it, and given you're using ant it makes sense to use the companion product ivy.

[Mumfrey]

By the way, if you're not familiar with it, here's one of my own ant scripts which shows how to pull down the deps with ivy.

The key parts are the retirieve step (which dumps all the requisite jars in "libs") and this funky shorthand which then adds all the fetched libs to your build-time classpath.

The actual dependencies are just specified in a simple xml file and transitive dependencies are handled automatically.

[ElgarL]

Because I currently have no dependencies as I'm testing out features of Sponge. I should not be required to use a build system or script to simply export a jar. As to Ivy, is that something new? Another download/install? More levels of complexity which are not needed. May be handy but not needed.

However I've always managed my own download/management of deps with Ant.

[Mumfrey]

Nope. Ivy's older than fire, it's kind of the companion product to ant and you should check it out. As I said, I'm a miserable old so-and-so and basically stick to ant+ivy for all my own stuff. I certainly wouldn't call it a layer of complexity, it's kind of the opposite, basically it removes complexity because you just say "download all this crap" and lo, crap is downloaded.

[ElgarL]

I just looked it up and really I have no need for it. I wrote an Ant script years ago to handle downloading deps and I've never had to alter since. I can download Towny onto a clean install of Eclipse with zero other requirements. Build a fully working jar without any need for anything else. Its how it should be. Simple and clutter free.

[Mumfrey]

That seems like cutting off your nose to spite your face. Things can be extremely simple, if you already acknowledge that you need to download deps automatically, then using a dependency manager provides that simplicity. Are you downloading everything using raw URLs or something? That seems to be more complex than adding a single jar to your project then just declaring the dependencies in a one place.

You've kind of lost me here because on the one hand you seem to want to keep things simple, but seem to be doing so in the most complicated way possible.

[Mumfrey]

I'll happily write your ivy configuration for you if you want, can you link your project's source or is it private?

[ElgarL]

I would rather write a simple one line in an Ant script to download a dep than install another app. Not sure if Ivy comes packed with Ant these days, but it looks like it didn't.

Also I download deps from my own server. To rely on anyone else hosting is suicidal.

I'm a minimalist when it comes to installations. I'd rather write a 10 line script than install some new program to do that for me.

[Mumfrey]

It's an ant plugin, so you just bundle the jar with your project, then import it in your ant script.

Would you at least consider seeing the other side here? It seems like a lot of your pain is effectively self-inflicted because you would rather do things a difficult way than a simple one. Whilst I completely agree that forcing someone to use maven or gradle is not something we should do (let's face it, people using scala will want to use sbt for example), managing dependencies by hand is not something anyone wants to do. There are off-the-shelf solutions to deal with what you want.

Ivy is a simple, repository-agnostic dependency manager that is really easy to set up and will remove all your pain going forward, and as I've said I'll happily set it up for you now so that there's zero effort on your part to make the transition (in fact the ulterior motive is that it'd give people a good go-to example for others who don't want to use maven or gradle) :smile:

Ivy supports everything from full blown maven repos to simple "get lib from this url" declarations, and effectively all you'd do (assuming you have your deps declared in your ant file, with urls to same) would be to move that list into a separate file (also xml).

I think everyone on this team could engage with your point if there were a cogent argument behind it, but at the moment I don't see how we can come around to your point of view when there doesn't seem to be any benefit to your approach, and you are in fact making things a more complicated than they really need to be.

[ElgarL]

My point is, I don't need anything extra to handle it. I have Ant built into Eclipse and it does everything I need. I could install Ivy to do that then I could install something else to do whatever and before long you have 100 different little programs installed, each one doing one little thing.

I have ONE install at the moment. I run Eclipse and I'm done. Nothing else. No other plugins and nothing else to add complexity or to go wrong. If someone else wants to build any of my programs all they need is Eclipse. Nothign else and nothing to configure. No dozens of different plugins for x y and z.

These days everyone is after the latest this or that. Got to have the most fashionable build tool or plugin to auto something or other. In the field of programming the old adage of KISS still applies and should ALWAYS apply.

An API when added to a build path should not be exposing unknown/undefined objects to an IDE.

[jeffreykog]

You're a bit overreacting now. There is no 100 little programs.

It's just the ivy jar for ant which will download all the files for you that you download manually from ant in the towny ant script. If you still don't want to touch ant you can always download all the dependencies by hand or using ant.

[ElgarL]

Its called an exaggeration to explain a point.

At one point it was only Ant you needed to install. Oh and the Ant plugin for Eclipse. Now its Ivy, or its Maven and the Maven plugin and whatever else.

I don't need nor want any of it. A decent IDE and an API is all I should need.

[octylFractal]

An API when added to a build path should not be exposing unknown/undefined objects to an IDE.

What do you mean by that? You mean things that don't have the javadocs you'd get by using ivy?

[Zidane]

@kenzierocks

He means the fact that using just the API jar will not work as Optional is from guava which is a class not in the API jar. A build system obviously rectifies this as it has guava auto-download and the end user is none-the-wiser.

[Mumfrey]

I have to agree with jk-5.

My point is, I don't need anything extra to handle it. I have Ant built into Eclipse and it does everything I need. I could install Ivy to do that then I could install something else to do whatever and before long you have 100 different little programs installed, each one doing one little thing.

As I said, you don't "install" ivy, because it's an ant plugin you just bundle it with your project, then in your ant script you import the jar. Nothing extra needs installing at all.

I have ONE install at the moment. I run Eclipse and I'm done. Nothing else. No other plugins and nothing else to add complexity or to go wrong.

This would still be the case, the only change would be you could also install IvyDE (an eclipse plugin) which would read the classpath out of the ivy file automagically. However that's optional and just having the ant script use ivy isn't another step at all, you'd still just run the ant file like before.

These days everyone is after the latest this or that. Got to have the most fashionable build tool or plugin to auto something or other. In the field of programming the old adage of KISS still applies and should ALWAYS apply.

I completely agree, but this is where your argument doesn't make sense. KISS says use the simplest solution, but your solution is more complex than need be.

An API when added to a build path should not be exposing unknown/undefined objects to an IDE.

How is downloading the jars automatically any different than bundling extra classes opaquely within the API jar?

Its called an exaggeration to explain a point.

For reference, this is called "hyperbole", and it's perfectly valid I agree.

At one point it was only Ant you needed to install. Oh and the Ant plugin for Eclipse. Now its Ivy, or its Maven and the Maven plugin and whatever else.

For the record, both ant and maven are now bundled with eclipse, so if this is your argument then really you should just use maven, since it's all built in an and handles 100% of everything you could want to do. I mean, don't do that, because maven sucks, but the it doesn't work as a point in your argument is all I'm saying. :smile:

I don't need nor want any of it. A decent IDE and an API is all I should need.

Indeed, I agree. And all Ivy does is automate what your ant script is already doing in cleaner and simpler way.

[ElgarL]

Because with a shaded API jar there are no downloads. You are not downloading anything because the API is providing everything it should be. There are no additional deps, nothing extra to install or configure Works straight out of the box on a vanilla IDA install.

[Mumfrey]

Indeed, so does my proposed solution. Your ant script is (as you mentioned) above, already grabbing everything you need, so this would just automate that and grab the transitive dependencies as well.

[ElgarL]

As I already said, I'm not using any script at the moment.

Do as you please but I'm not porting/coding for Sponge as it currently stands.

[Mumfrey]

As I already said, I'm not using any script at the moment.

Sorry I'm really confused. Are you using ant to download the dependencies for your project or not? You mentioned earlier

However I've always managed my own download/management of deps with Ant.

And I'm not proposing to change that.

Do as you please but I'm not porting/coding for Sponge as it currently stands.

I don't see why you would write off an entire API based on wanting to stick to something which is only going to become harder to maintain going forward. And this isn't about "as we please". I'm honestly trying to help you here and I don't see why you are so combative about the whole thing. Feel free to drop onto IRC later and we can chat about it.

As I said above, I think we could all get on board with your idea if it made any sense, which given that there's a simpler solution literally sitting in front of you (did I mention already that I'll do this for you so you don't need to lift a finger, and it doesn't change any of your workflow) it just doesn't seem to.

I'm sorry you feel that way but if you don't want to accept help when it's offered openly then I'm genuinely baffled.

[Tzky]

Mumfrey, if i got him right this is not about the way dependencies are added but about the fact that additional dependencies are needed. He wants only one dependency (spongeAPI), as he thinks an API should be the only dependency for plugins.

Maybe staff can discuss this further and present a decision when done?

[Mumfrey]

I understand what he wants, I'm just trying to get to the root of why he wants it. Since it represents a fairly hefty deviation from best practice (and, let's face it, sensible practice) there needs some solid reasoning for undertaking it.

Here's an analogy:

Let's assume you're a car manufacturer. You make and sell cars in a standard format, with standard controls that people understand. The standardisation is what makes the car safe and usable to people who know how to drive cars in the normal way.

Someone comes to you, and says. "I've taught myself to drive by laying in the drivers seat back-to-front, and looking where I'm going using mirrors. Can you mount mirrors in the back of the seat so that I don't have to add my own every time.

Of course, the first question you ask isn't "what shape should the mirrors be?", you ask "why don't you just try facing forwards". The user points out that they don't like using cruise control and ABS, but you point out that you don't need to use cruise control or ABS to operate the vehicle, you just need to sit facing the right way and can ignore those features which make driving slightly more streamlined if you want a more "hands-on" driving experience. Just sitting facing forwards is the simplest option, in fact it's simpler than your current solution because you don't have to fit your own mirrors every time you buy a car, and can in fact see more because you can see out of the windows as well.

The user protests that they like their mirrors, and could easily gain the "seeing out of the window" functionality by adding more mirrors, but you point out that if they just face the other way then they get this all for free, and moreover will have a nicer driving experience in general in the future.

The manufacturer hasn't actually said "no" yet, but trying to convince the user to adopt a simpler, easier solution should be the path of first resort, because really you can see that the user is actually making things harder for himself unnecessarily, and the features he proposes to suit his use-case may harm someone else (maybe people will get mirror glass in their back, or just be plain confused by the mirrors in the seat and think that they too need to drive facing backwards).

Of course, if nothing comes of the discussion, and the user is determined to preserve their somewhat eclectic and challenging mode of driving, then eventually the manufacturer might relent and make the mirrors-in-seat an built-in optional extra on some models. But the duty of care is to scope out the simple solutions first, the one which provides the greatest benefit to the end user.

It's also probably worth waiting for @kashike, who opened the issue to weigh in with his/her opinion too, since the discussion has thus far been dominated by @ElgarL and I discussing Ivy, and we haven't actually gotten to the original use-case of the OP.

I personally dislike being "forced" to use any one solution as well, so I can completely see where @ElgarL is coming from, but as I mentioned above, there are fundamental contradictions in his argument which bear further examination before going any further. Indeed, much as I hate to admit it, Maven is actually probably the best choice for his specified goals (just install the IDE, no extra steps) because it's built in to absolutely everything and it's stupidly easy to use. The only drawback with maven is that its repository architecture isn't pluggable so he'd have to muck around with it if he wanted to stick with the "everything on my server" setup. Ivy on the other hand is super configurable and could easily be a drop-in solution for his use case with pretty much zero effort.

We'll wait and see what other opinions surface anyway.

[boformer]

For simple plugins, a build system is a total overkill. Especially if you are new to Gradle/Maven.

I agree that when you publish the source code of a plugin, a build script is a must-have.

I don't see the benefit of a build system for little (private) experiments. For example, I have a "SpongeTest" plugin in my IDE to test the example code I'm writing for the SpongeDocs.

I was working on a wizard to simplify the setup of the gradle script, similar to the LibGDX setup (they also require gradle).

[Mumfrey]

For simple plugins, a build system is a total overkill. Especially if you are new to Gradle/Maven.

If you read the backlog, this point is not in contention. However, using a dependency manager like Ivy is super simple and doesn't have any of the issues which a shaded jar would have.

As for your other points, please keep general unrelated discussion out of the issue thread, otherwise it's likely to wander off topic. This isn't about whether using a build system is good or bad, it's about whether a shaded jar is needed to support those not using a build system (which it isn't).

[boformer]

@Mumfrey Ivy is just another tool that stops people from starting plugin development. I think a shaded API jar simplifies the process.

[Mumfrey]

Again, that's off topic though, because we're never going to actively encourage people to start developing without using a build system, it's just silly. It'd be like encouraging people to start writing a plugin without an IDE. The "it's just simpler" use-case is and always was off the table. The active point of discussion is people with existing workflows and the simplest way for them to integrate SpongeAPI into that.

In this instance, the active platform is ant, and plugging Ivy into ant is a no-brainer.

[ElgarL]

@boformer is most definitely ON topic. @Mumfrey you keep talking about it being a simple case of 'adding ivy' or whatever else you define as simple. Adding anything is the WHOLE point. It should not be required to add anything to write a plugin for sponge. Sponge should be the complete API and all you need. Requiring developers to add anything just to use the API is making it less appealing and harder to use.

KISS (Keep It Simple, Stupid), and about the simplest you can get is adding Sponge to your build path.

[Mumfrey]

Adding anything is the WHOLE point. It should not be required to add anything to write a plugin for sponge. Sponge should be the complete API and all you need.

But this is the point which I dispute. Why should it be that way, it doesn't make any sense to me. I kind of get where you're coming from but still don't see the use-case. New users can use the SDK so it doesn't benefit them, so that only leaves people with existing build chains and I already pointed out that there's a better solution for that in your particular case.

Requiring developers to add anything just to use the API is making it less appealing and harder to use.

But appealing to whom? New users aren't going to be put off by the SDK, and since you're using ant already there's a straightforward answer for you too. I guess what I'm asking is, who is this for? I still don't get it. It's a solution in search of a problem.

[ElgarL]

I guess you never will then. I've done all I can to explain it.

[Mumfrey]

I'm trying not be closed-mined about this, but you haven't made an argument yet which makes it clear why this is necessary.

[ElgarL]

I've said all I can. Some can see the reasoning some can't, but I have no other way to explain it.

[Mumfrey]

I must apologise then, I'd like to help but I really can't see how.

[Mumfrey]

Would either of these options work for you:

• We add a task in the API build.gradle file to emit a fully shaded jar, which would allow you to build it and host it on your server, which is what I assume you want to do ultimately.
• We do that, but I set up my jenkins (ci.liteloader.com) to also build that shaded jar and give you a place to download it from.

If either of those are helpful (or both), then I can set that up. It keeps the fully shaded jar off the official Sponge repositories but gives you somewhere to get what you need.

[ElgarL]

Its a nice offer and I believe a shaded jar is the correct way to go, but I feel it needs to be an officially available option. I think it should be the default option over a build system favoured one.

[Mumfrey]

I can't see us encouraging people to avoid best practices though. If you want the artefact available for your own purposes then then above offers stand and you can just poke me on IRC (I'm online when the M in my name is uppercase).

For ordinary/new dev we have the SDK and that will basically always be the recommended option. You may wish to await confirmation from a project leader but I'm almost certain they will say the same thing, sorry.

[ElgarL]

I've never seen a 'best practice' defined to require a build tool.

[ConnorStone]

Don't mean to bud in, but if I can help, I definitely see the point on both sides. I believe what @ElgarL is trying to say in that SpongeAPI should be as simple to setup as bukkit, if the programmer WANTS it to be.

For example, if you wanted it simple:

• API jar to the build path
• Profit

Where as if you were managing a complex set of multilevel dependency projects, such as Sponge and some of the bigger plugins, you might opt for a much more complex build system with control over every aspect of the build and all the dependencies.

I see @Mumfrey 's side of this, we don't necessarily want to discourage learning to use build scripts, but either way I believe anyone, novice or "old timer" should have a choice, even if it means they are being very simple minded about a specific project case scenario. I don't believe this is about Towny, but that it should be simpler for people if they want it to be.

We should remember for people who have never heard of Maven, Gradle and Ant, it's all a bunch of mumbo jumbo and they would rather start off simpler. I remember I first learned java cause I wanted a bukkit plugin, and the tutorials were so easy to follow. For new coders, a build system would be too much.

[ryantheleach]

Would bundling the dependencies in a shaded jar hurt anyone that doesn't want to use it?

Does it hurt the sponge project at all (apart from having to serve the extra bandwidth required)?

If not I don't see why it shouldn't be an option. Just a discouraged one.

Also btw @ElgarL not having a maven repo for towny is an utter pain in the ass for anyone that was developing bukkit plugins that tried to interact with towny. Instead of just specifying a repo location, a name, and a version people would need to hunt down the url and either download it or make special provisions in order to download towny to fulfil their towny dependencies.

Whilst I dislike having to chase down extra dependencies, I hate the idea of having excessively large API and the same libraries used included and shaded into everything more...

[jacklin213]

Tbh can't we allow both options of either a build script or a shadded jar.

Especially for a rapidly changing project like Sponge, not everyone can keep up dependencies. Correct me if I'm wrong but for the people who haven't used build scripts before they don't have to go out of their way to learn about build scripts before trying to develop their idea into sponge.

I quite like the idea of what @ConnorStone said

• API jar to the build path
• Profit

If we could support both that would be fantastic

[ST-DDT]

The none shaded jar should be the recommended one. But there should also be a shaded one.

The main issue with shaded jars are tutorials on YT or other platforms. Beginners will follow their lead. Videos outrank every textual documentation we could provide on our page. So we must catch all those tutorial guided people and show them a video and/or documentation on the download page. So the user starts thinking its easier using a build tool. They may not listen to it the first time but after a while they will at least give it a chance when they downloaded the API for the third time.

Beginners count the steps and how complex they sound not the real work. And especially for those beginners who have neber heart of java or programming at all a shaded jar is a must have.

• Download IDE
• Download Shaded-API
• Create Java-Project
• Add API to class path
• Type the 1 class code provided in the video
• Feel happy that there are no unexplainable red lines
• Export jar to the mods folder
• Test plugin and be happy and proud of yourself

No XML or similar stuff because it looks like dark magic and you hardly take the time to explain it in a tutorial. Bukkits plugin.yml is an exception to this rule as it is quite simple. (No brakets and only 4 lines you can write on your own)