Add support for global plugins/universal addons

mrjustaguy commented 4 years ago

Describe the project you are working on: For this the specific project doesn't matter, what matters is that I'm using the same addon for several of my projects.

Describe the problem or limitation you are having in your project: I see it useful for people (like me) who use a ton of the same addons across different projects because it would speed up things if it auto loaded the addons from some directory i set for the engine (much like you set the engines default project directory)

Describe the feature / enhancement and how it helps to overcome the problem or limitation: It would be a feature that would allow for automatic addon placement in a new project, thus speeding things up for people who use a ton of addons and make a ton of projects.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams: The idea is simple, the user can make a Directory that you label as the universal addon directory in the editor settings (much like you set a directory for default project location), so for every new project when it is created the editor will automatically put the addons (or it could ask which addons you want to put from the directory in when the project is created)

the pseudocode would look something like:

Grab editor setting "Universal addon" (or some such name)
Gets the new project directory
(Upon project creation, optional) popup a list of addons from the directory that can be toggled on/off
(in case of 2 implementation) When ok is pressed copy all of the addons toggled on
Grabs the addons, places them in the new project directory as an addon

If this enhancement will not be used often, can it be worked around with a few lines of script?: It can be worked around by making a directory where you store addons and you can just copy them over to the project, so it's not that big of a deal, this would just shave off a few steps making this quicker.

Is there a reason why this should be core and not an add-on in the asset library?: Because it's an improvement to the add-on/asset library and one couldn't make an add-on for Godot that would do this (I'm fairly sure atleast)

arkology commented 4 years ago

MarcusElg commented 4 years ago

Hasn't this been suggested lots of times already....

mrjustaguy commented 4 years ago

I guess it has been suggested before.. but a different path to doing so..

Xrayez commented 4 years ago

From what I've heard (mind you, just rumors), core devs planned a different approach with those plugins to be treated as regular projects (which could be nested), going for the approach inspired by the node-based system.

maciek134 commented 4 years ago

@akien-mga asking you since you explained it nicely in the closed PR - is @Xrayez right about the approach this should take? I'd love to have this feature and I'll be happy to create a PR, but I'd like it to be mergable. Could it be as simple as treating all projects in HOME/godot/plugins as global plugins, or did you have something else in mind?

aaronfranke commented 4 years ago

so for every new project when it is created the editor will automatically put the addons

What about a way for addons to be used globally, but not installed in each project? For example, with https://github.com/godotengine/godot-git-plugin, it doesn't really have a purpose of being inside the project aside from that being the only supported option. This addon could be deleted from the project and the project would still work fine, so it doesn't need to be copied into the project if there was a system to allow it to exist elsewhere.

me2beats commented 4 years ago

could global addons just to be located somewhere, say in some "global plugins dir" then the user could specify what directory to use as global plugins dir and after that when creating or loading a project Godot enables these global plugins?

also maybe there could be some API methods like set_global_project_dir enable_global_project_plugin etc

About UI implementation: It could be a plugins list in Global Settings — Plugins, like we have it in Project Settings — Plugins or implemented as one Plugins List (for both Project and Global plugins), but then it shouldn't be located in Project Settings menu.

I believe the ability to have global plugins would be handy.

But on the other hand, there are several concerns:

1) safety of other projects. for example, if a global plugin can delete the content of projects, then when such a plugin is activated, each open or loaded project will be deleted. in this case, one need to be as careful as possible when using third-party projects that use global plugins.

2) it may increase projects coupling, this can be compared to a singleton pattern, which, if used incorrectly, easily turns into an anti-pattern and only creates more problems.

silentorb commented 4 years ago

One benefit of global plugins is, depending on how they were implemented, they could remove the pressure to add nice-to-have plugins into the Godot core.

For other game engines such as Unreal and Unity, many features that would be considered core are actually plugins that are installed and enabled by default. Over the years Blender has transitioned to a similar structure.

If Godot supported the ability for someone to zip a collection of plugins with the Godot executable, that would allow a variety of different Godot installations to choose from, both official and third-party, similar to Linux distros.

That way a beginning developer could download an installation with a hundred enabled plugins while the Godot core remained light and minimal.

That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable.

Calinou commented 4 years ago

We discussed this yesterday in the Godot proposal meeting. We acknowledge it's a desired feature and could be useful, but at the same time, we're wary about uses of the "global add-ons" feature for add-ons that should clearly be stored within the project. In other words, your project might break if it relies on global add-ons but other people in your team forget to install them. We don't want to require people to scour the Internet manually to find add-ons before they can start working on a project.

To summarize, we think "global add-ons" work best for editor-only features that aren't related to your project, not features that will be exported into the final project.

That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable.

Global add-ons aren't a strict requirement for moving core features to add-ons. See godot-interpolatedcamera3d for an example. If this add-on was to be used as a "global add-on" by someone, this could cause projects to break unexpectedly when switching PCs or working in a team. To give a counterpart to Node.js, it's the same issue as using npm install -g for project-specific dependencies :slightly_smiling_face:

For what it's worth, the Node.js community is attempting to fight this problem by disallowing the use of globally-installed modules in the project itself. It will take a while for them to fully solve this issue, though.

Two-Tone commented 4 years ago

No, but to a large number of users they won't know that the once standard features are now add-ons they have to manually download and enable for each project, which will create some ugly, negative word of mouth that will be hard to fight.

On Fri, Oct 2, 2020 at 3:37 AM Hugo Locurcio notifications@github.com wrote:

That would also give core developers the option to convert some core modules into plugins, which in turn would allow game developers to toggle those modules without compiling a custom executable.

Global add-ons aren't a strict requirement for moving core features to add-ons.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot-proposals/issues/831#issuecomment-702602267, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBEIS5QRSMM2L7EDH3S7U3SIWGOBANCNFSM4M7Q55OA .

esijg commented 4 years ago

In other words, your project might break if it relies on global add-ons but other people in your team forget to install them. We don't want to require people to scour the Internet manually to find add-ons before they can start working on a project.

I am all in for global add ons, but as seems to be the consensus here, we need to be careful. I've had that exact same experience with PureData back in the day. I even have problems viewing my own old Pure Data projects as I've lost some of the global plugins and can't find them online anymore. :/

dark-penguin commented 3 years ago

Indeed, it would help greatly if "go do that in GDNative" was a better experience! And I agree that it should only be for the "editor plugins". The plugins that have anything to do with your project must be included in your project.

Thoughts about the implementation:

Of course, plugins that go into your project should be in res://addons/ . But "editor-only" plugins should definitely not! This way, you don't have to specify specific addons in your .gitignore - this folder is in your project for a reason. And that makes a clear difference between one and the other.
I would suggest that editor-only plugins are accessed with addons:// , or even better - with addon:// , which is the path to this addon! This would eliminate a few other problems. First, an addon won't have to hard-code its own name in every load(""), which makes it easier to rename addons (and fork them). If your project is an addon, then this should be as simple as usual with "res://" , but for addons. Second, you could put LICENSE and README.md in your root folder, which is good for Github, and users downloading your addon won't have to worry about checking the files and not installing a very important part of your addon because it would overwrite their own files. Since this addon does not go into your project, it does not have to be in res://uniquename anymore !
Downloaded "non-global" addons should be kept in your "local library". When you start a new project, you can open your "local library" and choose which of your global addons to enable, and which of your non-global addons to install. I would even suggest that the "fourth editor button" should not open the Asset Library, but the list of your local addons instead, and there should be a button to "Download more...". Now it's the opposite way - you go to the online library first, and from there you can go to your "Plugins". This really should be the other way around - with this, you will access your local library more often than the online library.
If you update a "non-global" addon in one of your projects, the newer version is kept in your local library. It will not automatically update in all your other projects, but when you do update it, you won't have to download it again.

With this, we wouldn't even need "Favorites" implemented in the Asset Library, because you would simply download them all.

dark-penguin commented 3 years ago

Turns out this issue is already there since long ago: #1205 Redesigning the plugin system is not so simple, so the discussion is ongoing.

(EDIT: this is slightly different, but it's mostly the same general issue)

SIsilicon commented 3 years ago

I would like to add my two cents to this proposal. About dependency issues when working with git, perhaps a link (soft or hard) can be used without copying the data in every project you open. If another user doesnt have the add-on, they can choose to sync their global addon repo with the global addons in the user's project. Anything else would have to managed per team. Then another thing to consider is cross platform support, 'cause git doesn't like Window's soft links, and Linux and Mac OS don't appreciate users making hard links to folders from what I've read.

pjgf94 commented 3 years ago

I made a template project folder with all the globals add-ons, then instead of using the project manager new project, i copy the template folder & change project name in project settings, you can also make a custom project manager.

lewiji commented 3 years ago

I was wondering about this earlier, I was thinking perhaps the Project List window could have an AssetLib tab added next to the Templates tab. Templates are cool but I'm not sure how often they're used practically (forgive me if they're actually frequently used!) so it feels like there could be more functionality added there. Also the distinction between assets and templates can be a little unclear (some assets in the AssetLib, for example, don't put anything in the addons folder, and often seem more like templates or examples, but ones you can just drop into an existing project rather than having to create a new one).

Perhaps the Project List could also be a kind of package manager or cache for the AssetLib, with an extra tab next to Templates. You could download assets you frequently use (or add via local repo for tooling you've created), which saves the zip to the Godot config folder, and then have a checklist in that tab that lists your "cached" assets, and allows the New Project dialog to add and extract the selected 'default' assets to new projects. You could then have further fine-grained control over which default/non-default assets you want to include in the New Project dialog with a similar checklist there.

I've never dug into the Project List code before so I'm unsure how feasible it is. I think it'd be a nice addition though. It seems like it would be relatively doable with reuse from the editor AssetLib dock, the Plugins checklist, etc, but I'm not sure how coupled they are to the editor vs the project list.

Frontrider commented 3 years ago

There ARE addons that need to be in a global folder by their nature. I use these to make the editor align with my workflow (a modified version of the second one): https://github.com/Frontrider/Godot-Component-Browser https://github.com/remorse107/vertexsnap

And they should really not be inside my project, as these are pure editor only plugins.

Not gonna argue that a heightmap terrain might need to be in the project, but a pure editor only plugin that has no right to be accidentally packaged into the app?

Xrayez commented 3 years ago

I've started to create many sub-projects (sandboxes) for my main Godot project (to speed up development and not to worry about breaking other things in the main project), and I've been recently using symbolic links/junctions on Windows to share addons between them, and so far this kind of approach works fine. But having support for global addons is still must have for editor-only plugins.

Those addons are still versioned in git repository, but files are committed only once rather gazillion of times between projects, and there's no way for those plugins to go out of sync this way.

Xrayez commented 3 years ago

I really think that the new GDExtension feature must have a way to share code between multiple projects. With C++ modules, you can do this by simply recompiling the engine with the classes you need, and all those classes will be available in all projects statically. This is not currently possible to achieve with GDExtension/GDNative (at least I haven't received response at Godot contributors chat to confirm this), and I wouldn't want to constantly install and/or recompile the sources in order to use new classes for each new project I create, like sandboxes/prototypes.

That said, I really believe that support for global plugins must be implemented for a feature which acts like/mimics the built-in features 4.x (GDExtension). This would increase the adoption rate of GDExtension over C++ modules.

Again, this is must have to implement if GDExtension aims to deprecate/replace C++ modules fully. Otherwise, feel free to ignore what I've said.

aahmyu commented 2 years ago

To summarize, we think "global add-ons" work best for editor-only features that aren't related to your project, not features that will be exported into the final project.

It's been a year since this comment, did this get implemented?

I find it weird to install a "Command Palette" plugin in my project folder.

Calinou commented 2 years ago

It's been a year since this comment, did this get implemented?

No, work hasn't started on this yet.

I find it weird to install a "Command Palette" plugin in my project folder.

Command palette is a built-in feature in Godot 4.0 :slightly_smiling_face:

aXu-AP commented 2 years ago

Reading this thread I'm disoriented. There are 2 different topics being discussed here, which should be 2 different proposals. Both of them good ideas, mind you.

Is this about editor-wide plugins (not in project folder) or about managing plugins (make adding them to projects easier)? The original issue seems to be about latter, but most comments seem to discuss the other. The title is ambiguous.

From op:

It would be a feature that would allow for automatic addon placement in a new project, thus speeding things up for people who use a ton of addons and make a ton of projects.

Frontrider commented 2 years ago

Reading this thread I'm disoriented. There are 2 different topics being discussed here, which should be 2 different proposals. Both of them good ideas, mind you. Is this about editor-wide plugins (not in project folder) or about managing plugins (make adding them to projects easier)? The original issue seems to be about latter, but most comments seem to discuss the other. The title is ambiguous.

You get this because "automatically adding plugins to a project" covers those domains. A plugin added automatically to the project acts somewhat like a "global editor plugin" with how Godot handles this at the moment.

L2750558108 commented 1 year ago

So when will this be resolved? It's been two and a half years...

Calinou commented 1 year ago

So when will this be resolved? It's been two and a half years...

There is no ETA for implementing proposals, as contributors work on a best-effort basis. In any case, this will only be tackled after 4.0 is released, since it's in feature freeze now.

SlugFiller commented 1 year ago

One thing not mentioned is that if you're using a plugin in multiple projects, and you are also actively developing said plugin, it is currently very difficult to keep it up to date in all the projects where it is used. In general, a plugin should NOT be in the source tree, especially if version control is used.

Since @Calinou used npm as a (bad and incorrect) example, I will too. When you have a dependency in node.js, you only specify where to get it from in the package.json. It does not become part of the source tree, and if you use Docker than it doesn't even become part of your directory tree.

The way addons are implemented currently is really the worst of all worlds. It's a recipe for tons of code duplication, and versioning nightmare.

Calinou commented 1 year ago

When you have a dependency in node.js, you only specify where to get it from in the package.json. It does not become part of the source tree, and if you use Docker than it doesn't even become part of your directory tree.

Once you run npm install, you have the dependency's code sitting in node_modules/. It just happens to be a folder that is usually .gitignore'd, as npm install can essentially recreate the same directory structure (with lockfiles).

Nowadays, there's a strong preference in the Node.js community to declare developer tools dependencies within package.json (such as linters and formatters), instead of using global installation. These tools are then run locally using npm scripts, which use the version exposed in node_modules/.bin first if available. So this is actually the opposite of what's proposed here, technically 🙂

SlugFiller commented 1 year ago

@Calinou

Once you run npm install, you have the dependency's code sitting in node_modules/.

Unless, as I've mentioned, you're a sane person using Docker, in which it doesn't, it only sits in the transient virtual drive, until such time as you run the build script again. And you really need that extra layer of safety ever since version 6 or 7, where they added non-disablable auto-prune, and actually ended up deleting a package I was developing without asking for confirmation, which only really makes npm a really bad example for anything related to good design past versions 4 or 5 (Although it's a great example as to why open source projects have a non-warranty clause in their license). As it is now, it's only usable in a container or with -g.

By contrast, node.js itself has always checked the entire directory ancestry for any parent with a node_modules folder in it, so you can just as easily put /node_modules (or c:\node_modules) and use it in all of your projects. Which I've actually done. So even -g is not needed.

The fact remains that npm at least auto-pulls dependencies, which Godot doesn't, so even if you accept tips from npm, Godot's current incarnation is still an incomplete system. If you do add a sharable dependency list to a project, there becomes largely zero reason not to de-dup installations, by installing all addons in a common folder, but only activate a selected set. So either way, it's just another point for central installation folder over installing in the project folder.

The one part where I feel there's an issue, is if there are scripts or resources in the addon that might be used at runtime, e.g. scripts for custom controls. The system would need to know that these need to be added to the project. However, I feel like the current way is simply "wrong" in the opposite direction - Instead of including too little, it includes too much. There's no reason why an import script should be packed in your project's runtime. IMO, files actually being added to your project from an addon should be opt-in. As in, by default, the addon only works in the editor, and specific scripts and resources needed at runtime are marked as such by the addon's configuration.

Even better if a different version was applicable for runtime vs edit time, so various features that only make sense in the editor (e.g. Saving whether x and y editing are locked together or set to be separately editable) could be stripped even from custom components, but that's a separate subject.

ghost commented 1 year ago

I have a plugin that does this, but you still need to get it into each project separately.

gregcsokas commented 1 year ago

I made a similar proposal for this problem #6350 (and since mine was a duplication I closed it) My proposal is a little bit different than this one, instead of putting information about the global add-ons in any project file, I propose storing global plugins in the config folder, and storing the addons in the editor-settings.tres

Frontrider commented 1 year ago

I think (looking at other discussions) this issue is how godot handles "packages" at this point. That might be a better angle to approach this.

I think this is the bare minimum work for what was collected here (brainstorming, not trying to ration what should be done)

add a new "addons" folder into a global location (could be the editor data folder), and show what is in there when I open the plugins panel in the project settings.
Include the scripts from the folder of the enabled plugins

At this point we do not need to download the same one package over and over again. This can at least give us the option to copy/symlink custom addon to that folder and make it available.

Push a local addon into the "global addon folder", this might be omittable because this feature can be written with GDScript if we can access the editor's folder. (say have a plugins:// path to access that folder)
Store some package identifier in the project properties for each addon/asset pack/template we installed. This needs a repository format that is more readable than "one random number".

This last one is also a site issue, unless we/you are willing to "recycle" an existing repository format to store zip files.

I'm objectively biased with this sentence, but the maven format is already suited to handle any zip files, and provides a verbose enough package naming scheme that can give you all the information you need to identify what you are about to download. Plus you get enterprise level support for custom repositories out of the box. (Gradle also has a "subsystem" that can pull c/c++ dependencies from zip files stored in maven).

Using an existing format, would allow Godot to just boot up an existing repository software, and have at least that portion work out of the box.

Any name/repo format would work here, as long as it is human readable.

SlugFiller commented 1 year ago

Rethinking about this, this whole thing may be overthinking. It's not necessary to have a global addon location. You just need a way to share addons between projects without relying on OS features that may or may not be present in any given OS. Basically, a way to specify an addon folder that is not a subfolder of the project folder.

Enabling arbitrary absolute or relative ("..") path would be suitable. Alternately, the nodejs method could be used, of searching up the folder tree for a specifically named folder (e.g. "godot_addons"), and allow using any addon from inside that, as if it was in the addons folder inside the project. This would be more "secure", because it means a project can't arbitrarily include files from anywhere on the drive (although a third-party project can still use FileAccess from a tool script). It also makes it easier for a project to auto-detect available addons without needing for the user to browse to addon locations.

This would be sufficient to achieve de-dup, which is enough to make addon development and use less of a hassle for anyone working on more than one project. The process of installing an addon to a shared folder can be left up to the user, since it's a one-time action.

Frontrider commented 1 year ago

Rethinking about this, this whole thing may be overthinking. It's not necessary to have a global addon location. You just need a way to share addons between projects without relying on OS features that may or may not be present in any given OS. Basically, a way to specify an addon folder that is not a subfolder of the project folder.

A proper package management is significantly cleaner then random folder linking, and also solves other issues that we have now as well.

SlugFiller commented 1 year ago

@Frontrider What other issues? Can you be more specific?

When you say "package management", it makes me think "dependency hell" which is probably not where we want to go. At the very least, this issue doesn't mention the case of an addon depending on another addon.

In fact, it specifically states "use a ton of the same addons across different projects". The primary goal is de-dup, and/or have a preset to enable a whole bunch of addons for a project at an action cost no higher than that of creating the project in the first place.

And, as additional benefit, it would prevent "I use it often" as an excuse to request engine bloat for something that absolutely can and should be implemented as an addon. The main issue isn't that we need package management (the AssetLib is already as friendly as necessary for discovering and installing an addon), it's that people generally work on more than one project, and some addons are useful for so many different projects, that people mistakenly think they should be engine features.

Frontrider commented 1 year ago

When you say "package management", it makes me think "dependency hell" which is probably not where we want to go. At the very least, this issue doesn't mention the case of an addon depending on another addon.

What I see, is that I don't need to either duplicate code I can use, or tell the user to install something else for the full experience.

In fact, it specifically states "use a ton of the same addons across different projects". The primary goal is de-dup, and/or have a preset to enable a whole bunch of addons for a project at an action cost no higher than that of creating the project in the first place.

There was a good reason why the talk picked up NPM. You need that kind of tool to de-dup properly, without having to manually check things. This is inevitable as the engine grows.

(the AssetLib is already as friendly as necessary for discovering and installing an addon)

The asset lib is in maintenance mode already, only receiving critical fixes. This was the attempt for a replacement, and if you look at the commits, (Calinou, not intending to say anything bad with this) they are very sparse. That is why I suggested to try alternatives where only a client is needed. https://github.com/Calinou/godot-asset-library-laravel

And the personal specific case was seeing how could we package level editing tools, and it is either essentially copy-pasting them between repositories, or install by hand.

SlugFiller commented 1 year ago

@Frontrider

There was a good reason why the talk picked up NPM.

It was used as an example of anti-patterns. NPM is indeed very, very terrible. As is Yarn. Both no longer provide cross-project de-dup, instead relying on their own per-project JSON as Word Of God(TM). But the basis that node.js provides, and NPM and Yarn horribly screw up, is pretty good.

That is why I suggested to try alternatives where only a client is needed.

Bringing up NPM/Yarn again, they are 3rd party tools (Especially Yarn), that are not part of node.js. But the node_modules system that is built in to node.js allows them to work. And it allows de-duping without using such terrible tools, as well. (The tendency of node programmers to use many one-line packages in their dependencies instead of just adding one line of code to their project, does make it harder). What we need is the infrastructure, not the solution.

And the personal specific case was seeing how could we package level editing tools, and it is either essentially copy-pasting them between repositories, or install by hand.

This is very similar to the OP's request. Namely, simply being able to specify the location of the level-editing tool as outside the project folder already allows you to de-dup it, and use it across many projects. You don't need a specialized tool for that. If it is multiple level-editing tools (in separate addons), then you either need a loadable "preset" that applies multiple addons to a project at once, or a way for a single addon to do that instead. The prior is likely simpler, though. Something like loading a text file or JSON containing a list of folders that have addons in them. Then, you only need to prepare this file once, and the rest is just a couple of clicks.

gregcsokas commented 1 year ago

Okay, I just realized I need a little bit different thing like original post.

I have a few plugins/add-ons that I want to use everywhere, In a new project, in an old one, I want them when I open the editor. And I don't want to copy them based on some checkboxes.

few examples: -godot-wakatime -WAT or Gdunit -Aseprite wizard -Color preview

Currently if you add a new addon/plugin to your project it will be stored in the project.godot file like this

[editor_plugins]

enabled=PackedStringArray("res://addons/ColorPreview/plugin.cfg")

With the global addon/plugin thing this will happen, There will be an addons folder in the /home/<user_name>/.config/godot/ folder next to the feature_profiles, projects, script_templates, text_editor_themes

and a similar section in the editor_settings.tres file.

asset_library/global_addons/path = "/home/<user_name>/.config/godot/addons"

[editor_plugins]

enabled=["ColorPreview", "WAT", "AsepriteWizard"]

After the implementation in the project settings/plugins there will be two sections, first is the global plugins then the local ones, or the global plugin's settings will be available from the editor/editor settings menu

if your turn off something from the global section, that would never load after that until you don't enable it again. but if something is enabled in the global section, it will load automatically with every new and old project, with every project template. And you don't have to copy things from anywhere to multiple places.

I'm not sure I can implement it, but I can give it a try for it.

Snailedlt commented 1 year ago

+1 This would be very useful for plugins like godot-wakatime

Snailedlt commented 1 year ago

If you need inspiration as to how this could work, take a look at popular coding IDE's like Jetbrains IDE's (IntelliJ, Rider, Webstorm, PyCharm, etc...) and VSCode.

All of these allow you to have global plugins (all plugins are global by default). Jetbrains IDE's also allow you to have project specific "Required Plugins", and VSCode allows you to have Workspace Recommended Plugins, which is similar but not the same, as it does not enforce you to install those plugins, but it recommends them to you in the "Plugins" tab when you're using that project.

Imo the most important one for Godot is the "Required Plugins" which we already have. It would however be nice to also have global plugins, and "Project Recommened Plugins" (Like VSCode has) for plugins that may be useful, but not required for the project.

gregcsokas commented 1 year ago

I would like to do simply as possible, I'm thinking about plus one checkbox in the Create New Plugin popup with the Global plugin label, if you check it on, you can create a global plugin, and the editor can save this into the plugin.cfg, so if someone wants to create a global plugin there will be an option for it. And if you want to install one from the asset store, the engine can handle it easily.

(all plugins are global by default)

I don't think it's necessary, If you have 5 projects as I have, and 3 of it is 3D, you don't need the SmartShape2D everywhere.

This workspace recommended plugins sound interesting, but I have to say it already exists, under the name of the asset store category. Since there is no option for making a decision about the type of the new project. I mean there is no button for different project types like in unity ( 2D, 3D, 2D (URP), 3D (URP), VR, AR, Cinematic etc), you can create any kind of project after the create a new project window. But in the asset store, you can filter for 2D tools, 3D tools and Tools in general. So I can say maybe in the next step, we can implement this Workspace recommended plugin stuff, but there are other requirements for it I guess.

Yeah, I'm using JetBrains products for 6 years now I guess :) There are some really cool features in that IDE. And this Required plugin is a really handy one, but, if you want to use plugins like the godot-wakatime, is that really a required plugin? Honestly, I don't think so, and if something is required for your project, that probably is not a global one I guess, but I think is more like a discussion about the semantics/philosophy of the "how to handle a plugins" topic.

If you think I misunderstood something please explain it further.

arkology commented 1 year ago

I think first of all this proposal https://github.com/godotengine/godot-proposals/issues/6307 should be implemented.

Frontrider commented 1 year ago

(all plugins are global by default) I don't think it's necessary, If you have 5 projects as I have, and 3 of it is 3D, you don't need the SmartShape2D everywhere.

Global does not mean that it is also always enabled. We can most likely piggyback on the existing system for addon enabling/disabling for this, so I see it as a non-issue. I don't think all global plugins must be enabled permanently, but that option would work for things like git support.

All of these allow you to have global plugins (all plugins are global by default). Jetbrains IDE's also allow you to have project specific "Required Plugins", and VSCode allows you to have Workspace Recommended Plugins, which is similar but not the same, as it does not enforce you to install those plugins, but it recommends them to you in the "Plugins" tab when you're using that project.

The categories I can see for godot are "editor plugins" and "game plugins". Those are the types that I've seen so far. One adds a new editor feature that does not make sense in the game (eg on godot 3, the metadata inspector plugin, or my scene browser), the second one is say a node library that needs to be exported with my game.

I believe we can make the global ones to be the "editor only" plugins. That is code that is better to not live inside your project folder, and may not even be required by the project itself. Wakatime and the git plugin are excellent examples for this type of plugin. Search it up in a menu, open, install, ok, rinse and repeat. Setup a git submodule. Continuous copy-pasting. I work with IT automation, I see this "infinitely repeated manual install" as a significantly larger issue than I think you do. This can mean that you "have to reinstall your editor for each project", if you could have enough plugins in your "normal workflow". I see that Godot could easily be re-tooled for different people in your team (level editor, designer, animator etc), but right now the entire project must have everything, everyone needs to enable everything otherwise you might break someone else's flow with a single commit. This feature would be extremely powerful, but right now it is easier to just keep everyone in separate tools.
The "workspace/game" plugin in godot's current context could clearly mean that I want this to be exported. This also goes back to @gregcsokas 's "SmartShape2D" example, I can see that as a "workspace plugin", because we need to export it. Not having editor plugin code in exported games is also a request, and this solution could solve that too.
There is a third type which is a plugin that does both, and that is the difficult one to figure out. As a "worst case scenario" the global plugin could download/copy the code that needs to be exported into the project's folder. This would also prevent the project from breaking because of an uninstalled addon. (the code you depend on is still in your project!)

I don't think this is a perfect setup, because I really don't want to see the code for most of the plugins I use as libraries in my workspace, but it is already better then what we have now. I don't think that this separation is arbitrary. Your workspace becomes "what is here, is what should be in your game", while you also gain the power to customize your editor to your personal needs without any hassle.

I think first of all this proposal https://github.com/godotengine/godot-proposals/issues/6307 should be implemented.

We could even try to distribute addons as their own PCK files as well. This might be what we actually want.

The PCK can be loaded from any location, so we already have that feature covered
IF the pck gets their own resource paths, then we can help to isolate them, dependencies can be included without/with less risks. (can avoid file overrides) I do see one possible issue, which is dependency duplications increasing the binary size. But trading file size for increased reliability is one I'd count as good tradeoff, and I personally also don't expect it to be significant enough to matter. If it becomes a problem, you can still try to take the sources for those addons and include them by hand. I don't believe this matters for the editor, at this stage that can be ignored.
The dependencies themselves can be removed from the sources, don't need to be in git
This format is already used by godot for data to be loaded at a "later date", right now it means runtime patches and mods. Why couldn't it be loaded by the editor too?
Distribution is also a non issue, you can have standardized and hardened github/gitlab workflows for automatic publication. I never checked if we can load a pck into the editor from gdscript, if we can then we can make a prototype for this.

Unloading a PCK is gonna be a problem for the editor (unload the pck to disable the addon), but if we have them in their own namespaces then I think it would be solvable.

rsubtil commented 1 year ago

Seems like this issue got pretty confusing because it was interpreted in both ways, as @aXu-AP put it:

Is this about editor-wide plugins (not in project folder) or about managing plugins (make adding them to projects easier)?

I've recently opened a issue meant to tackle the first interpretation, and it seems that the most recent discussion has been about this interpretation as well, so I'll continue in that direction.

In that regard, I believe this feature needs to be as simple as possible, without any unnecessary complexity on top. Essentially, I'd say it's just a matter of loading plugins outside of res://, which will require #6307 as @arkology pointed out.

We could even try to distribute addons as their own PCK files as well. This might be what we actually want.

I don't think this is a good idea, it only adds unnecessary complexity without any significant advantage. This is also a feature that's still not very mature, as noted in https://github.com/godotengine/godot-proposals/issues/2689.

Frontrider commented 1 year ago

I don't think this is a good idea, it only adds unnecessary complexity without any significant advantage. This is also a feature that's still not very mature, as noted in #2689.

Then don't. I don't care if what I throw in does not stick :) (it usually does not stick, but lets me find what does). #6307 can be useful anyways.

Also, always remember that complexity purely depends on what tools you have available. If you have a tool that can reliably do a very complex task with minimal to no input than that task is now easy to do.

rsubtil commented 1 year ago

Also, always remember that complexity purely depends on what tools you have available. If you have a tool that can reliably do a very complex task with minimal to no input than that task is now easy to do.

I didn't mean complexity of the task, I was referring to the complexity of the solution.

Frontrider commented 1 year ago

I didn't mean complexity of the task, I was referring to the complexity of the solution.

Thinking a bit more: This should not have any internal complexity, and we even have a "convention" for it. The only "requirement" is #6307, and only to make sure that things can't clash with one another. IF we rely on the addon convention inside the PCK (gdscript is stored under /addons/<addon name>/) then it can't clash already.

What needs to happen is for the editor to load any *.pck files inside the project folder. Unloading should not be necessary as we can just treat it the exact same way as native plugins. (the existing convention)

More on topic: We could just have the "global" part as a folder that is added to your project as a separate resource root, we load all gdscript/gdextension/pck from that folder for every project.

SlugFiller commented 1 year ago

@Frontrider First, you say that as if #6307 is not complex. Long-needed, sure, but complex.

The bigger issue is that if you need to place pck files inside your project folder, it's not significantly better than placing a full addon directory inside your project folder. It certainly doesn't help with the issue of actively developed plugins being used in multiple projects.

The way #6307 can really be used, is by mounting a folder from anywhere on the harddrive, into the addons folder, within the editor. This takes care of de-dup, but not of large addon presets. The latter can be taken care of by mounting the addons folder itself. But without a merging mount, the latter would be an all-or-nothing solution.

Allowing addon folders to have more complex locations within the project, or parallel filesystems (e.g. an addon:// filesystem, where every subfolder at any depth can be an addon), would take care of that last corner.

me2beats commented 1 year ago

The idea of the proposal is to just copy project to each newly created project. My thoughts are:

can it be just solved with local project templates? https://github.com/godotengine/godot-proposals/issues/1481 You would just configure some template, add all you need (including plugins and other assets like Kenney materials or so), and set the template as default. Then when you create new projects, the template folder content are just copied
What if I want some godot 4 plugin be in every godot 4 project I open (including old projects and third party projects)? The proposed feature doesn't solve which makes it not 'universal'. It can add global plugins to all the projects when they are opened tho but there is the third problem
Should the universal add-on contents copied to the project be read-only? If not, should the addons modification be tracked? Say you decide to make some changes to the global plugin X, or change it to some new version so you modify some files of the global plugin X or just replace the plugin folder with a new version. Now perhaps you want all projects to update this plugin X in their directory. In this case, should plugins have some hashsum. And if I remove the add-on from global add-on folder, should it be removed automatically from all projects as well (when you open them)? Or ask what to do, or do nothing?
The size of you have plugins with some meshes and textures you may not want to bloat each project in file size

—

I imagine global plugins not to copy contents to each project folder. Instead it can use and refer to the plugin from the global plugins folder. This is challenging tho:

the global plugin contents (for example scripts) should be read only (the user should not be able to modify the script from the editor; or the notification/warning should appear like "you are going to modify the global add-on resource, this will affect all projects, continue?"
if the plugin have some resources (textures, audio etc), that is supposed to be used in the game, then these resources should be copied and be kinda read only I guess and have modification info (is the resource modified in the project) **
some API should be added, like is_global (to easily check if the plugin is global)

** this makes that things a magnitude order harder to implement I guess, so as Calinou said it would be better if global plugins would be editor only. but how to know if the plugin is "editor-only" and will not affect the game? It doesn't write anything to res at least I guess. Maybe it can be even a separate EditorGlobalPlugin class that disallows to for example add new custom nodes This means for example plugins like BetterCommandPalette or BetterAnimationEditor, ProjectStats or EditorDebugger could be global, and TerrainPlugin and KenneyTextures couldn't

SlugFiller commented 1 year ago

Trying to modify a "global plugin" should indeed have a warning attached, but it should still be possible. The main reason is dogfooding. If while using a plugin, particularly one that you developed yourself, you find a bug when it's used in a specific way, being able to fix it in-place, in the current project, instead of trying to create an MVP and fix it there, is incredibly helpful for development.

And, if you're fixing a bug in a plugin, you want the fix to be available to all projects. So auto-updating is a must.

The unfortunate part is that this can't be limited to editor-only plugins. Other forms of plugins also benefit from dogfooding. It's a lot easier to develop a plugin inside a project that uses it, than "blind". And developing it in multiple projects helps make sure it actually works as a plugin, that is, that it doesn't depend on anything specific to a single project.

Calinou commented 1 year ago

Trying to modify a "global plugin" should indeed have a warning attached, but it should still be possible. The main reason is dogfooding. If while using a plugin, particularly one that you developed yourself, you find a bug when it's used in a specific way, being able to fix it in-place, in the current project, instead of trying to create an MVP and fix it there, is incredibly helpful for development.

If you have to do this, I think it's best to copy the add-on locally and have the local add-on override the global add-on in a given project. This makes sure your patched add-on is committed to version control, as you don't want to lose your work if you move your project to another machine (your patched version may not be available anywhere online).

godotengine / godot-proposals

Add support for global plugins/universal addons #831