Better custom EditorPlugin nodes

[zatherz]

Nodes created by the add_custom_type function in EditorPlugin have the selected script assigned to it when adding. This makes them almost useless, as you can only use the functions defined in that script in other nodes.

This is completely different from other nodes and makes node addons pretty much useless/much more annoying to use.

[Zylann]

@willnationsdev it should be noted that registering in ClassDB also means people making plugins should all prefix their classes to prevent collisions^^"

[willnationsdev]

@Zylann Yeah, that would be necessary, unfortunately.

At this point I think I've sorted out most of the bindings and core changes. It's just a matter of updating the Editor, CreateDialog, and EditorHelp / DocData classes now (and the GDScript compiler).

[reduz]

As I mentioned before.. I still don't think the proposed solution is good. Plese check with me before attempting something that will most likely be rejected.

On Feb 7, 2018 20:05, "Will Nations" notifications@github.com wrote:

@Zylann https://github.com/zylann Yeah, that would be necessary, unfortunately.

At this point I think I've sorted out most of the bindings and core changes. It's just a matter of updating the Editor, CreateDialog, and EditorHelp / DocData classes now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6067#issuecomment-363893711, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z2xIYRcs0BFDqTyiFwqlhQWTqjEZtks5tSgIVgaJpZM4JejbZ .

[willnationsdev]

@reduz Well, I've been trying to take into account the concerns you previously laid out.

I am not implementing any sort of multi-script system. I am also not making scripts masquerade as ClassInfo objects in the ClassDB. All I've done thus far is attach a script (via RefPtr) to the inheritance hierarchies in the ClassDB. The Object class will only ever use a "backup_script" whenever the regular script assignment attempts to infringe on the backup script's inheritance hierarchy.

[reduz]

Oh I see.. what is the use case for this then?

On Feb 7, 2018 20:32, "Will Nations" notifications@github.com wrote:

@reduz https://github.com/reduz Well, I've been trying to take into account the concerns you previously laid out.

I am not implementing any sort of multi-script system. I am also not making scripts masquerade as ClassInfo objects in the ClassDB. All I've done thus far is attach a script (via RefPtr) to the inheritance hierarchies in the ClassDB. The Object class will only ever use a "backup_script" whenever the regular script assignment attempts to infringe on the backup script's inheritance hierarchy.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6067#issuecomment-363900920, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z2yn68Iy6AgAKJHZ4qImH4UBAm5skks5tSghSgaJpZM4JejbZ .

[willnationsdev]

The use case is for when someone wishes to introduce a new type of Node to the engine using an EditorPlugin, but they do not wish the scripted type of that node to be editable (they want the interaction with it to mimic an in-engine type). So, for example, if I attempt to get_script() on a node with only its backup script, then it will return null. If I attempt to set_script(null), then the script gets set to the backup script. If I attempt to set_script(a_script), then it will only successfully assign the script if the new script is in the inheritance hierarchy of the backup script.

For UX, I plan on making the script icon appear in the editor as a transparent icon if only the backup script is present. In that case, the button to add a script will still be an "add a script" button rather than a "remove script" button, and clicking it will prefill the text with the name of the type (although, I may need to make it prefill with the script path if the user changes the type of script away from GDScript / VisualScript, etc.). Once a different script has been assigned, the transparent icon would become opaque again. Clicking the script icon in either case would take you to the active script source code (the backup script source code if no other script is present).

[reduz]

I have to admit I don't really see the benefit of this, so again what's the specific use cases you are thinking of?

On Feb 7, 2018 20:46, "Will Nations" notifications@github.com wrote:

The use case is for when someone wishes to introduce a new type of Node to the engine using an EditorPlugin, but they do not wish the scripted type of that node to be editable (they want the interaction with it to mimic an in-engine type). So, for example, if I attempt to get_script() on a node with only its backup script, then it will return null. If I attempt to set_script(null), then the script gets set to the backup script. If I attempt to set_script(a_script), then it will only successfully assign the script if the new script is in the inheritance hierarchy of the backup script.

For UX, I plan on making the script icon appear in the editor as a transparent icon if only the backup script is present. In that case, the button to add a script will still be an "add a script" button rather than a "remove script" button, and clicking it will prefill the text with the name of the type (although, I may need to make it prefill with the script path if the user changes the type of script away from GDScript / VisualScript, etc.). Once a different script has been assigned, the transparent icon would become opaque again. Clicking the script icon in either case would take you to the active script source code (the backup script source code if no other script is present).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/6067#issuecomment-363904934, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z26oeOMT4HpjzDsX8eygMSeUhbTgVks5tSgurgaJpZM4JejbZ .

[Web-eWorks]

The high-level use case is to allow plugins (EditorPlugins, the PluginScript/SubModule system) to define 'fool-proof' custom node types. Currently, the custom type system is a thin factory for "script on a node", which has numerous workflow and usability issues if you are attempting to build new node types, the least of which is the complete inability to easily, intuitively, and effectively extend the custom node with gameplay code (contrary to built-in nodes). A more pressing issue is the inability to create custom nodes that inherit from other custom nodes.

Some specific use-cases are to create low-level nodes: perhaps a marching-cubes based voxel GridMap implementation for destroyable terrain, ala Minecraft or any other terrain system using marching-cubes/dual contouring; custom Control types (circular menu container, as implemented in many modern games, any type of low-level Control that's just high enough to be outside the engine spec); and any place you want to build a plugin that registers new nodes. In each of these cases, you usually want to add a custom script to handle gameplay code, and have it be separate from the "implementation" code (written in GDScript/GDNative).

I think what's failing to be communicated here is that the goal is to make custom nodes behave like engine nodes without having to write an engine module and compile against it.

This improvement is not necessarily driven by a specific feature (though there are many), but rather by the rationale of "the current way is backward, flimsy, and tedious". If implemented fully, this system would allow an extremely powerful method of extending the engine's built-in types from GDScript / GDNative without needing to recompile the engine. From a design standpoint, this is literally a straight improvement over the current system.

If you want a particular use case after all of that: the entire plugin / AssetLibrary ecosystem. That's the scope of what benefits from this change.

[willnationsdev]

Ok, so here is an explicit example:

I have a plugin I am working on, godot-skills, which introduces a variety of new types of nodes:

Effect: nodification of a Functor Targeter: similar, but just finds and collects other nodes Skill: combines hierarchies of Targeters and Effects to find nodes and apply effects to them all SkillUser: "owns" several Skills

I would want each of these nodes to be things that users can create and add to a scene directly. However, each and every one of these nodes are designed to serve as a base type. They don't have a lot of functionality out of the box, and must have a deriving script placed onto any node they would otherwise be involved with. For use cases such as these, it can be useful to create a node from which you don't want the user to be able to remove a particular set of scripted functionality. For example, I want to prevent users from placing a non-Targeter-deriving script onto any Targeter node they add to their scene. This is because there is instant functionality that Skills get from having Targeters as children (similar to Area2D and CollisionShape2D nodes).

The current workflow requires that users...

1. Add a custom type node 2a. Remove the script attached to it and click the add a script button OR 2b. Set the script property on the node
2. Find in the dialogue window where the EditorPlugin is keeping the custom type script you wish to derive.

This system would allow you to replace all of this with just:

1. Add a custom node type.
2. Add as script (which automatically starts deriving the custom type script)

Users would be able to dynamically add their own custom types that derive the in-engine types (for simpler usability). It would also reinforce the security of the plugin's functionality by enforcing the presence of the scripted functionality on the node in question. Currently, users can easily just remove the script and break things (not desirable). Furthermore, if someone wants to "clear" the script on the node, then that node shouldn't lose it's custom type functionality (this is a big one). It should revert to having the custom type script.

[willnationsdev]

Ok, so an update. I currently have the following.

• The ClassDB can now add and remove custom types directly.
• The EditorPlugin has similar custom type methods that add and remove not just the custom type, but other editor-related data such as the icon. These methods are connected to a new signal which is emitted whenever the EditorPlugin is toggled active/inactive.
• GDScripts can now properly inherit directly from the names of custom types to inherit from the script.
• The scene dock will create a transparent script icon when the current script matches the node's custom type script. Clicking on it will open up the custom type script in the editor. While in this state, the "add a script" button is displayed rather than the "remove a script". If selected, this button will prefill the "Inherits" field with the name of the custom type script (if non-null).
• Any attempts to remove a script will cause the custom type script to be re-assigned as the script value. As such, in order to remove the script, you must first clear the custom type script value with set_custom_script(RefPtr());.
• The create a node dialog properly shows inheritance relationships and abstract nodes for custom types.
• The project.godot file is populated with all custom types so that executed games can set them up in main.cpp prior to the rest of the game starting.

Remaining tasks to be completed...

• Allow users to generate class API docs for custom types.
• Fix a separate bug altogether that prevents users from editing the Inherits field in script_create_dialog.cpp.
• Find a way to dynamically add and remove custom types from the GDScript global map/array *
• Implement custom type support in VisualScript

* I actually have a few questions about this.

First, I only see add_global_constant / _add_global in the gdscript_compiler.cpp file without any means of removing globals from the GDScriptLanguage class. Does it just get completely re-initialized whenever singletons are added or removed? How are global identifiers edited dynamically? This would be so that users can do MyNode.static_func() without having to load the script into a variable first.

While I'm at it, I feel like Godot's GDScript could benefit greatly from the introduction of namespacing, especially as part of this change. We'd have to find the right delimiter to flag them, but you could do away with concerns about name collisions from engine and plugin types when making your own scripts. For example, assuming \ is the delimiter of choice...

extends Node # an in-engine type.
extends \ # triggers an error if left alone, but would prompt a dropdown of the plugin API names for auto-completion as well as the names of any scripts in the current project that are not part of a plugin.
extends \MyPlugin\ # triggers an error if left alone, but would prompt a dropdown of the scripted types that exist for the plugin
extends \MyNode # extends my_node.gd somewhere in res://
extends \MyPlugin\MyNode # extends my_node.gd somewhere in res://addons/[my_plugin_something?]/

func _ready():
    print(\MyNode.CONSTANT) # would print CONSTANT in the current project's my_node.gd script

Anyway all of this ^ is just ideas being thrown around at the moment. If namespacing were introduced, you could even have scripts automatically be associated with a name based on their file name using the resource_saved signal in EditorNode. They WOULDN'T necessarily be custom types (in the sense that you can't remove the script from the object), but they WOULD be accessible by name alone which would be extremely useful and intuitive. Scripts could even override the default auto-naming (filename.capitalize().replace(" ", "")?) by providing a namespaced title at the top, e.g. \MyPlugin\mynode. Now suddenly "\MyPlugin\mynode" is a global identifier for that script.

[willnationsdev]

Here's a snippet of my progress thus far.

[Zylann]

@willnationsdev please don't choose \, it's already used for escaping and multiline. ., : or :: look better.

[willnationsdev]

@Zylann I planned on using the delimiter as a prefix for any nodes that get added to the project as well. So then, for example, a script at res://container.gd wouldn't want to have collisions with the actual Container class. If we used a period (.), then it might look like this.

var engine_type = Container.new()
var script_type = .Container.new()
var plugin_script_type = .MyPlugin.Container.new()

Anyone have objections to that kind of format?

Edit:

That might run into issues within derived class functions though that call the parent's method within them:

func _virtual_method():
    ._virtual_method() # parent stuff
    # my stuff

Hmm...does that pose a problem for constants and/or variables? If not, then we can just assume that parentheses means it's a super function and otherwise it's a possible namespaced identifier.

[Web-eWorks]

Honestly, I'd hold off on the namespacing for right now - it could be introduced later, and I'd rather get the current system in as soon as possible. We would also need a really well thought-out system for automatic name registration, possibly extending to a reworking of the entire addon organizational structure, which is a fair amount of work and beyond the scope of this issue.

Personally, I'd rather the name for an 'auto-registered' script be controlled by a declaration in the file, i.e.

register <NAME> extends <PARENT>

or a similar syntax based on the current extend <NAME> prolog that's already present for GDScript.

EDIT: I also wanted to ask how assigning scripts deriving from a parent (say Spatial) to a custom child (say Sprite3D->CustomType) is going to be handled? This already works for engine types, and restricting assigned scripts to be derived from the custom type itself is not going to work out very well.

[willnationsdev]

@Web-eWorks Fair enough. I'll work on perfecting the current API and then I'll open up a new Issue for namespacing once the PR is submitted.

[Zylann]

If I understand well what @Web-eWorks means, I still have this question myself: how will you get around the fact an Object can only have a single script? How will they be... "scriptable"? Because apart from that, registering to ClassDB is nothing more than a global identifier in terms of use for programmers and designers.

[willnationsdev]

@Zylann What I've done in my fork is create a custom_script property for Object. It only ever becomes significant if a value is assigned to it, but if a script is assigned to it, then it serves as a backup script / constraint on what the main script property can be. Clearing script will set it to be custom_script, and if you assign a new Script to script, then it will only be assigned successfully if the Script instance derives from the custom_script. And all of this is managed by Object behind the scenes, so as far as the engine is concerned, Objects still only have 1 actual script.

The ClassDB content just has a secondary map of custom types that get registered, mapping the script itself and some other info to the name you've assigned it. It then provides that name and its inheritance hierarchy with related inheritance methods (get_class_list, is_parent_class, get_parent, etc.). And this is done as an opt-in flag for some of the methods. get_class_list will now take a bool flag where it only includes custom types if you pass true as a second parameter. This is how it prevents breaking compatibility with the rest of the engine.

[Zylann]

@willnationsdev so, with a system that makes scripts behave like builtin classes, what's preventing users from wanting all their scripts and addons to be considered first-class citizens like the builtin classes, and drop the "old-school" way?

[Web-eWorks]

Ah. The way scripts currently work is that you can put a script that extends Node on a VBoxContainer, and it still works just fine. If the current implementation breaks that behavior for custom types, there will be some issues - is there any way to keep the custom script's behavior while having a 'normal' script deriving directly from an engine type?

[willnationsdev]

@Zylann That's actually precisely why I want to introduce namespacing so that scripts and plugins' scripts can be safely integrated via names without colliding with in-engine types. There would be nothing preventing people from using only the new way and ignoring the old way. And I actually think that would be better overall. It's laborious to have to use an explicit path every single time.

@Web-eWorks Not sure if I'm following right here, but...

Custom types specifically work like in-engine types for inheritance. So if you extend from a custom type, it'll be as if you are extending from a completely different branch of the in-engine hierarchy. Ex. if I create a custom type MyNode that extends Node, I won't be able to put that script on a VBoxContainer just like I can't put a Node2D "on" a VBoxContainer. This would actually be the intended behavior.

Now, with the namespacing concept, you would ideally have named scripts that are NOT custom types. So, in that case, I could create a script that extends Node and put it on Node, VBoxContainer AND MyNode. Does that make sense?

[Web-eWorks]

So what I'm saying is if you create a custom type MyNode inheriting say MeshInstance, instance a node of it in the editor, and attempt to add a script inheriting Spatial to the MyNode typed node, will that work properly? It works properly with 'engine' nodes, and that's the behavior we need to emulate.

It should be namespace independent, as this is more an issue of making sure the custom type's script is getting called regardless of the point in the inheritance hierarchy of the 'user-level' script.

[willnationsdev]

@Web-eWorks Yes, that would work. Every "first level" entry in the ClassDB's custom_types map will have to link back to a type in the classes map, otherwise an error occurs and the custom type isn't created. This ensures that the chain of inheritance is preserved when moving from custom types back into the engine types all the way up to Object.

[Web-eWorks]

@willnationsdev Just to be sure - if you add a _ready method in MyNode, and a ready node in your user script that derives from Spatial, it will be called like Spatial._ready(), MeshInstance._ready(), MyNode._ready(), script._ready()? (Or whatever order _ready is dispatched in.)

[reduz]

I still stand by my view that this is unnecesary and confusing. This problem needs to be solved from the editor UI, not core engine.

[willnationsdev]

@reduz This isn't something that can be solved from the Editor UI alone though. Not if you want to control users' manipulation of the script on the object. It seems to me like that is a feature that many people are wanting to have setup.

Are you suggesting that the gdscript_compiler should reference content in EditorData in order to setup new global identifiers for scripted types, rather than tracking them all in the ClassDB and allowing the ClassDB to manage all of the inheritance information in the engine?

[Web-eWorks]

Certainly there's a part of it that needs to be solved from the perspective of the editor UI, but there's also a part that needs solving in the core. Adding custom types to the engine, actual types from the perspective of the ClassDB, is a very powerful and ultimately necessary feature. As far as confusing, I still maintain that it is no more confusing than the current system, and very probably less.

[willnationsdev]

@Web-eWorks I agree. To make the inheritance data be tracked in the editor would actually be much more confusing in my opinion. And it would result in odd workarounds as that type information wouldn't exist anymore once you run the game without the editor (which is why I actually had to start loading the information in project.godot in order to have it loaded into ClassDB on startup. No EditorPlugins to add the custom types was causing the GDScripts to fail compilation when running the scene).

[willnationsdev]

@Web-eWorks And to answer your earlier question, yes, that works properly.

In my test scenario, I have Node.gd which is extending GDSkillsEffect > GDSkillsAPI > Node, and the _ready methods implemented in each script get triggered in the proper order: API, Effect, Node.gd.

I am currently experiencing an infinite loop error of some kind though whenever two scripts both try to inherit from custom type scripts by name. Ex. if Node.gd has extends GDSkillsEffect and gdskills_effect.gd has extends GDSkillsAPI, then trying to modify and save gdskills_effect.gd or gdskills_api.gd will cause the editor to infinitely "load" and then crash. That's probably my biggest bug at the moment.

[Web-eWorks]

@willnationsdev Ahh, no, I'm talking about Node.gd extending Node, not GDSkillsEffect. If that works, then everything is good.

For example, here's a representation of the scene tree:

GDSkillsEffect
-> User Script:
  extends Node
  func _ready(): pass

[willnationsdev]

@Web-eWorks Ah, I see what you mean. Well, it SHOULD work, but I just tested it and it seems like my logic in Object::set_script() is blocking it for some reason, so that's a bug I'll need to fix. XD It's failing because it has detected that Node doesn't derive the custom type script (which is true, yay), but it shouldn't be failing in that case, lol.

[reduz]

@Web-eWorks As I mentioned, this is NOT something that needs to be changed in ClassDB.

I don't really think this needs to be solved in the core, as each scripting language handles it different. You can easily create a new class in Mono and it's going to work. For GDScript it may be a simple case of exposing a script file as a preloaded global and that's it. A one-size-fits-all solution, as proposed in the PR, is definitely not the way to go.

[reduz]

@willnationsdev I appreciate your effort, but this was discussed to no end already and the consensus is that:

• Adding a custom type is different for every script language, so hacking something in ClassDB, which is only really meant to register C++ stuff, is NOT the way to go.
• Before going to such lengths, we can improve the UI usability for cases where you want to extend the script of a custom node, so it just creates simple inheritance.
• In general, adding hacks to the core engine to handle corner cases is never the way to go. Core needs to remain hack-free.

[Web-eWorks]

@willnationsdev For reference, engine behavior is that any script extending something equal to or previous in the inheritance hierarchy can be added to a node. Thus, a script that extends Spatial can be added to a MeshInstance but not a Control or Node, as neither of the latter are or inherit from Spatial. This is behavior that must be replicated. (No pressure...)

@reduz But does creating a new class in Mono add it to the engine's inheritance hierarchy? Or the editor's Create Node dialog? Perhaps a new issue needs to be opened that concisely states what the purpose of this is, because I think we're not talking about the same thing here.

The point of the ClassDB modifications are to allow registering new types of Nodes (or Resources) without needing to recompile the engine. Not to add new custom globals in GDScript, though that's a tangential issue that @willnationsdev is working on in the same branch.

The point of this is not to add hacks, but to create a well designed system to extend the engine's types.

[reduz]

@Web-eWorks The engine inheritance hierarchy is for C++, not scripts. Trying to mix both of them is conceptually wrong to begin with.

Maybe the misunderstanding is that the create node dialog makes it look like a new class was added, so it may be a good idea to mark such custom types as clearly containing a script.

[reduz]

@Web-eWorks If you are using Mono, it will look transparently as if a new class was added, because it IS a new class. For GDScript, as classes are files by default, it can't be done this way so the way to do this may be to add a new global with a dedicated UI to set it up (as we have for singleton autoloads)... but this is still a script class, not an engine class.

[reduz]

@willnationsdev My whole point in this is that i think it's wrong to make it look as if you are adding actual new classes to the engine, because they are not. It's open to confusion. It's just a node with a script, so instead of hiding the script, I think the right thing to do is make it more visible in the create dialog.

[Web-eWorks]

@reduz Well, that's the thing. Node+Script is a Scene. The add_custom_type mechanism should behave like Mono, registering a 'new class', even if it's not technically a 'class' by whatever standard. If there's a way to do that without needing to touch the ClassDB, I'm all ears.

[reduz]

Also, @akien-mga suggested that in the scene tree view, we could hide the scripts of custom nodes, to avoid confusing users about them having created that script and edit it by mistake (but still show it in the inspector at the bottom).

In such nodes, adding a script will open the create dialog in an inheritance mode, so it will still be pretty transparent to the user workflow wise, while it's clear that it's a custom node and not part of the engine.

[Web-eWorks]

@willnationsdev Any objections to me opening a new, more concise issue specifically detailing the scope of implementation?

[reduz]

@Web-eWorks Node + Script is not a scene, nodes in a tree layout are a scene. You can also add custom resource types, not just nodes.

And no, it should not behave like registering a new class. It needs to be pretty clear that it's a C++ class with a script pre-added.

[willnationsdev]

@Web-eWorks Yeah, that's no problem with me. Please mention me so I get a ping if you do. Not sure if we need a separate issue though?

[reduz]

Look at it from another point of view. Your argument is to make it look to the user as if you were extending the engine types with new engine types.

My point of view is that you are extending the engine types with scripts, not with new engine types. And that the user is not stupid and needs to know this.

[Web-eWorks]

@reduz What I meant about Node+Script was that that mechanism was already covered via instantiating Scenes.

Maybe I'm mis-understanding the Mono implementation, but aren't Mono classes still Scripts?

[Zylann]

@reduz I mostly agree with you, also when you said that adding a script to a custom node would simply propose to inherit it. But how do you expect this to work with GDNative custom types?

[willnationsdev]

@reduz One of our goals though is to allow custom type scripts, these pre-assigned scripts placed on the C++ classes, to not be removable. That is something that can't be done in the editor context since you could always remove a script at runtime (unless you want to preserve that functionality and only secure a custom type script at design-time in the editor)

[willnationsdev]

I would certainly be open to an alternative method. Either way, the easiest implementation is to have the Object conditionally assign scripts to itself (as I've done in my implementation) by checking some third-party registration class (whether that's the ClassDB, ProjectSettings, EditorData, or who knows what else). It then becomes very easy for the editor itself to check with that third party and display things differently (also as I've done in my implementation). It wouldn't be too much work I don't think to move the registration to another context than the ClassDB though.

^ Would that compromise be acceptable? Otherwise, you have to micromanage every possible location in which Object gets a script assigned to check with the third party. That would be much more complex.

[willnationsdev]

The implementation I have done has been very clear thus-far as to which "types" are scripts and which aren't. After all, you can still clearly see and access the script associated with custom types in the scene dock in the video I shared.

[reduz]

@willnationsdev I understand what you are trying to do, but I don't think this can ever be made cleanly. It's hiding a complexity that has no reason to be hidden, in a way that will never fully work as expected.

@Zylann I think it's the same, but either a) the script is gdnative internal b) it uses a method internal to gdnative. I am not familiar enough with it honestly to tell.

[willnationsdev]

@reduz Are you saying that you don't want to allow any form of creating script-based constraints on Objects? Cause I could move the check logic into the Editor itself and that would give you design-time checking of the constraints just as easily (UI changes, as you say), but they wouldn't work at runtime, so people would still potentially be able to change things once the game is running. Would that work better for you? I'd still prefer to be able to check stuff at runtime though...

[reduz]

@willnationsdev what kind of script based constraints do you want to do?