Open willnationsdev opened 4 years ago
@Calinou While you are working on the Godot Asset Library Laravel implementation and work on adding asset version tracking, I request that you leave room for non-assets to also be version tracked (like these Packages) in the future. You might already be planning to do that, but just in case.
As for the "addon collection" thing, can this be achieved by publishing a dummy package that lists a bunch of dependencies? I think some npm packages do this. In any case, I'd prefer going for a simple solution :slightly_smiling_face:
I would agree, except that I don't know if we want typical addons supporting "optional" dependencies.
@Calinou Maybe we could make the underlying data in the database support both required and optional dependencies, and make addons and packages use the same configuration file to define themselves, and then have a flag set within that config file to say whether it is a package or not. And then, if it is a package, it will additionally search for and use an optional dependencies field which, if defined for an addon, just won't do anything. So the expectation would be...
Does that sound good? Any suggested changes? Do we like this idea at all?
Additional thought: I wonder if assets should be able to declare scenes/scripts to be automatically added as singletons when installing them. The name and path to the scene/script would have to be written down somewhere in a dictionary property. Removing the asset should also remove the singleton declaration.
This would make assets centered around singletons significantly faster to set up (which is good for prototypes or game jams).
@Calinou That sounds good to me.
It looks like there's someone else interested in contributing to improving the assetlib's frontend. Figured I'd let you know.
Also, do we know anything about if/when/how dependency management will be implemented? I remember there was an issue with a lot of discussion about it, but I don't think a consensus was every quite reached and idk if reduz has discussed those ideas on IRC at any point. I figured, if there is a data-driven way to extract the dependencies of an asset, then we'd want to be able to display those dependent asset links within an asset's webpage. However, we don't want Godot dependent directly on the Asset Library, so there'd have to be some way of referencing an asset that works both locally in Godot and in the Asset Library (fully qualified namespaces maybe? godotengine/godot#21215).
Also, do we know anything about if/when/how dependency management will be implemented?
No, it seems reduz is way too busy to discuss this right now.
@Calinou As for the "addon collection" thing, can this be achieved by publishing a dummy package that lists a bunch of dependencies? I think some npm packages do this. In any case, I'd prefer going for a simple solution slightly_smiling_face
Related -> #607
@nonunknown Out of curiosity, how does your proposal differ from this one?
it does not differ, it complements this one, specifying how plugins dependency should work :D
I need my plugin Y to not even start installing without plugin X installed and active.
That is, when I click the Install..
button (before the list of files to be copied to the project opens), I want Godot to find some deps.cfg
file in the addon folder (in the downloaded zip package), where I've listed the dependencies - plugins which can be downloaded from the asset lib, which should be installed and activated automatically, and only after that my plugin (Y) may be installed.
Could this proposal solve this?
Could this proposal solve this?
Yes, but nothing has been implemented yet.
@me2beats @Calinou There's also the fact that the mere decision of whether to implement dependency management at all hasn't actually been decided from what I've seen (unless there's a consensus in a proposal I missed somewhere). There's also this alternative proposal that reduz made a while back about creating a sub-project system and replacing addons with projects wholesale, so if you had dependencies, you'd literally just include them a copy of those dependencies directly in your project. So, there'd be a project tree just like there'd be a scene tree (except, you know, without the magic of scene instancing for projects).
Basically, I haven't seen any actual decisions made on how to move forward with resolving any of this mess, or any action taken to implement any of the proposed solutions, and it makes community-driven addon/plugin development stick in the mud and move like molasses.
Basically, I haven't seen any actual decisions made on how to move forward with resolving any of this mess, or any action taken to implement any of the proposed solutions, and it makes community-driven addon/plugin development stick in the mud and move like molasses.
I would like to see some kind of dependency management in Godot. However, we need to make every add-on a proper plugin before we can consider adding a way for plugins to declare dependencies. Having dependencies definable only for editor plugins isn't very useful. It's also likely we need to implement https://github.com/godotengine/godot-proposals/issues/554 first.
Another issue is that dependency resolution can get quite complex if you allow multiple dependency levels – and it can encourage npm-style micropackages to pop up, with all their issues. It would be possible to limit dependencies to a single level to prevent that, but it would also make the system less useful.
I would like to see some kind of dependency management in Godot. However, we need to make every add-on a proper plugin before we can consider adding a way for plugins to declare dependencies. Having dependencies definable only for editor plugins isn't very useful. It's also likely we need to implement #554 first.
Say there's kenney particle pack. Its content is just png images. Does this addon need to have dependencies? I think no. On the other hand, this addon itself can be a dependency. But should it have plugin scripts in such a case? I don’t think so. An asset config file might make sense though.
In my opinion the plugin / asset should just have
This system should only be supported for those assets/plugins, all files of which are in a separate asset folder: res://addons/some-plugin
This sets some restrictions for plugins, limits it's structure flexibility and may not be very convenient for the user, but on the other hand it would save you a lot of headaches, for example, when you need to update an asset to a new version, you just update the contents of the asset folder.
Another issue is that dependency resolution can get quite complex if you allow multiple dependency levels – and it can encourage npm-style micropackages to pop up, with all their issues. It would be possible to limit dependencies to a single level to prevent that, but it would also make the system less useful.
I don't see a problem with dependency levels - in my view it just has to be incompatible with the dependency system: each addon should be still installed in addons
in a separate folder.
several plugins in one addon should be prohibited (at least for the first time).
This does not mean that all plugins must follow this rule. If a plugin doesn't follow this rule, it just would be incompatible with the dependency system.
However, we need to make every add-on a proper plugin before we can consider adding a way for plugins to declare dependencies.
But should it have plugin scripts in such a case? I don’t think so. An asset config file might make sense though.
That's effectively what he's saying. With the current system, if a given addon has any global script classes, the editor can only toggle on/off those scripts if the addon in question has an EditorPlugin defined for it, thereby making it show up in the ProjectSettings' Plugins tab. And addons themselves aren't required to have any config file, as you mentioned, while plugins do require it.
Considering we are coming up on a compatibility-breaking release, I agree, it is important that we make critical changes that can establish new conventions, even if it doesn't create a full-blown dependency management paradigm. So, requiring a config file for every addon sounds like a good idea to me, and then we can have things in the Asset Library tab that allow you to "manage" the addons present in the res://addons/*
directories.
(Edit: actually, I guess, technically speaking, how it is arranged now with the ProjectSettings configuring the management of addon directories (currently as plugins), and then having a shortcut to open that management tab from the Asset Library is probably the best way to do that.)
I don't see a problem with dependency levels - in my view it just has to be incompatible with the dependency system
This actually sounds like a really good short-term solution imo. Until the sub-project stuff is implemented (which I would guess would be coming with some sort of 5.0 release), we really need some sort of barebones way of officially stating the set of dependent addons that a given addon requires, just so that it can ensure that those things are also installed.
We could adopt the solution the composer project uses. In composer the package library (or addon in this case) the package.json
file is optional. For dependencies that don't have their own package.json
but are required by a project file the package.json
of the project can define the package.
This is detailed here https://getcomposer.org/doc/05-repositories.md#package-2 and adopting this concept could make it so this change doesn't break BC.
Hi All! Recently I implemented my own plugin management system for my own modules. It mostly designed for process binary precompiled modules for third-party SDKs for iOS and Android. There are 37 production ready packages: analytics, ads, social network integration and so on.
The system has dependencies, autoloading gd wrappers, export hooks, uniform module settings for app keys and so on. It can be used as GUI addon and as CLI utility (for example when you want build your project with CI and don't want to store all binary frameworks in github repo. They can be automatically downloaded before your project was build.)
Full info and package list: https://drmoriarty.github.io/nativelib/ GUI Addon: https://godotengine.org/asset-library/asset/824 CLI utility: https://github.com/DrMoriarty/nativelib-cli
What do you think of it?
So a year has gone by and I'd like to ask about the state of this. I've heard rumours about "a new asset lib" and would've guessed that this would be part of that. We see a surge of addons and more are about to come it with more widespread adoption of Godot.
Can somebody please update the core team's current stance on this proposal? Thanks!
Can somebody please update the core team's current stance on this proposal? Thanks!
npm-style dependency handling is not planned for 4.0, as there are too many other things to work on currently. I think the plugin system in general will not change much compared to what we have right now (other than replacing GDNative with GDExtension).
The issues about allowing nested dependencies are still present and haven't been debated much by the core team yet.
I've heard rumours about "a new asset lib" and would've guessed that this would be part of that.
The new asset library is only a web backend – it does not impact how the editor uses assets. In fact, the REST API it exposes is fully compatible with 3.x
. However, there is no ETA for deploying it to production due to possible incompatibility with the hosting we're currently using (TuxFamily).
@Calinou thanks a lot!
For dependencies to work, a list of dependent addons would be required as a field in the backend. So this proposal would be about
Is there already some sort of epic/label with proposal for a new frontend?
I'm asking because I was tinkering with creating a new addon for a new asset lib frontend that could at some point be used as a base for a core implementation.
Is there already some sort of epic/label with proposal for a new frontend?
Not that I know of. As I said, the feature proposed here still hasn't reached consensus among core developers, so it may not be worth doing yet.
Ah okay, makes sense.
It just occurred to me. Godot 4.0 is gonna be a compatibility-breaking release, potentially for quite some time. Is there any plan to update/improve the addon/plugin ecosystem for 4.0, or is there gonna be some way to not introduce compatibility breakage for a new system so that it can be part of subsequent 4.x releases?
It just occurred to me. Godot 4.0 is gonna be a compatibility-breaking release, potentially for quite some time. Is there any plan to update/improve the addon/plugin ecosystem for 4.0, or is there gonna be some way to not introduce compatibility breakage for a new system so that it can be part of subsequent 4.x releases?
Feature freeze is imminent, so I'm afraid we can't make any significant changes now. The only change I'd consider making for 4.0 is https://github.com/godotengine/godot-proposals/issues/554, but I won't be working on that.
We expect some compatibility breakage to happen in 4.1 regardless, so we still have some leeway (though not as much as 3.x -> 4.0).
Sounds about right. My guess then is that we'd introduce some entirely new means of handling things like this, but set it up so that the old system is still "working", but deprecated (more than likely).
As I mentioned in #6704, this shouldn't be too difficult to implement if done through simply compressing the addon's folder into a .zip. Since Godot already has support for reading from zips, this should just be a matter of using this on the plugins, correct? (do let me know if I am wrong, as I have not looked through the source, and am relying solely on how I think it is done) NOTE: I am not suggesting it be unzipped, but that the zip be read
asset-lib
should fundamentally be a tool that manipulates and references text files.user@box|~/project $> godot --asset-lib-search "3rd person controller"
godot asset-lib: package(s) found: "3rd-person-controller-plug"
user@box|~/project $> godot --asset-lib-install "3rd-person-controller-plug"
user@box|~/project $> godot --asset-lib-require "3rd-person-controller-plug"
godot --asset-lib-*
manipulates and references.Here is an example of what that could look like:
[package]
name = "project"
version = "0.1.0"
[dependencies]
https://godotengine.org/asset-library/4.3/serve 3rd-person-controller-plug
https://github.com/godotUserJuan/godot-package-list custom-test-framework
Ultimately it should work in such a way that a beginner never has to touch the Command Line to manage packages, they may use only the asset-lib tab in the editor.
It should provide a CL interface so advanced and professional users may manage their packages in an automated environment or through a keyboard focused interface.
It should be text based because that keeps in line with Godot projects being hostable in and friendly to Version Control Systems such as GIT.
Edit: the more I think about this the more I realize that a package manager for Godot should probably be an external application and probably not be built into the editor at all. I could see it having an editor plugin giving the package manager presence in the Godot UI and some interop. However, in keeping with the spirit of Godot's newbie friendliness and simplicty, a package manager is added complexity that isn't really required of the platform.
Describe the project you are working on:
The Godot Next plugin: a collection of scripts that are useful in a variety of projects (utilities, editor tools, singletons, nodes, resources, etc.).
Describe the problem or limitation you are having in your project:
Godot Next is large enough that it has a small network of internal dependencies (ranging from 0 and 1 all the way up to 6 in the rarest case). No big deal if you grab the whole thing, but if you just want to solve a problem with one script, then you might have to deal with dependency management, even inside a single addon.
This is a big headache, both for me as the maintainer and for users who get interested and then very quickly frustrated/turned off due to needing to deal with resolving out-of-nowhere dependency chains.
Describe how this feature / enhancement will help you overcome this problem or limitation:
The best solution would be to increase the granularity of addons in Godot Next and introduce dependency handling to Godot. After all, some people want it all; some want just the most necessary bits. The choice of whether to make users manually pick out the parts they want from the whole or piece together the whole from a bunch of parts shouldn't be a mutually exclusive one. Both options should be available.
Right now we have 1 addon with a wide variety of classes. Some are related to others; many aren't. But, they are all a package deal. They are part of Godot Next's brand of, "a generic extension to Godot."
Rather than that, I would rather have N addons, one for each class or minor collection of classes, and have each tiny addon declare the dependencies that it has on other addons. I would then want to be able to declare all of these addons to be part of a thematic/topical "package" that users can use to mass select a collection of other addons. These could be defined as part of an AssetPackage resource type for arbitrary creation and storage. The Godot Asset Library and the EditorSettings would also be able to store these AssetPackage files for remotely distributed and locally shared "packages" of addons.
Furthermore, I would want to be able to browse and configure these "packages" of addons during the project creation and addon browsing processes to provide more concentrated value and discoverability to users.
If a new-ish Godot user opens up the Project Manager to create a new project, they should be able to view a list of popular Packages that they can install alongside their project. Rather than needing to create their project, go to the AssetLib tab, and then browse through hundreds of addons and install interesting ones one-by-one, they could instead see a GUI of addons to select which are batch installed at the end of the process and which can be selected/de-selected in groups by selecting/de-selecting themed Packages.
Users should even be able to create their own AssetPackage resources, either for local use (to auto-install a collection of preferred addons) or to upload and manage for the community (like what Godot Next could do).
A "Package" could then be...
Note that one difference between an addon and a package is that addons simply declare a set of dependencies while packages can have both required and optional addons associated with them, as demonstrated in the final mockup below.
Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
The final UI I imagine for this would be like a blend of the VS Code extensions UI and the Visual Studio Installer.
The former contains a list of elements where selecting any one element displays an editor-rendered Markdown README + any other custom data (repo web URL, rating, whatever data we can safely store and use, assuming it doesn't violate GDPR).
For reference, here is the VS Code extensions image in #139:
The latter maintains a list of selected editor components and will mass add/remove components from the batch installation dialog as users add/remove packages of functionality from the installation. Individual components can also be browsed and added/removed as desired.
Here is the original Visual Studio Installer:
I'm imagining that we could setup our own packages-over-addons setup with a VS Code-like README renderer. Something along these lines:
And this would be something that can appear both in the AssetLib tab and as a popup when users are creating a new project. Packages would be pulled from the Godot Asset Library as well as from local user files.
Describe implementation detail for your proposal (in code), if possible:
If this enhancement will not be used often, can it be worked around with a few lines of script?:
This enhancement is largely based on modifications to the Asset Library, the Editor's AssetLib tab, and introducing a new resource: AssetPackage. It also changes the Project Manager and is dependent on standardization changes in the addon ecosystem. It would revolutionize the accessibility and growth of addons, so yes, it would be used often.
Is there a reason why this should be core and not an add-on in the asset library?:
Tons of changes to NOT script-accessible stuff. See above.