godotengine / godot

Godot Engine – Multi-platform 2D and 3D game engine
https://godotengine.org
MIT License
90.79k stars 21.13k forks source link

Extensions for Godot #8505

Closed karroffel closed 7 years ago

karroffel commented 7 years ago

Extensions

After talking to @touilleMan and @reduz in Paris (it was awesome!!!) I'm writing down this proposal.

GDNative is a nice way to use third party libraries in your game or just optimize performance critical code, but it could also be used to add "extensions" to Godot (for the lack of a better name, "plugin" and "add-on" are already taken πŸ˜› )

Current situation

A GDNativeScript represents one class in a GDNativeLibrary. To have any effect, this script must be attached to any object. This is cool to write game code or wrap third party code in a class.

The current way to "extend" Godot is via plugins (or more precisely, "editor plugins"). These plugins can do a lot of awesome things, but they all only run in the editor. GDNative scripts can be used to make plugins as well, but they can't do anything that will have an impact on the running game.

Proposal

extension

Have a new project settings tab for "extensions". Extensions are GDNativeLibraries that don't need to expose any classes, they just get run and that's it.

These extensions get loaded at startup, in the editor as well as in a game.

Implementation of an extension

A regular GDNativeLibrary, but it contains a special function godot_native_extension_init and godot_native_extension_terminate that get called at startup and shutdown of Godot.

The C API (modules/gdnative/godot.h and modules/gdnative/godot/) will contain new functions, like `godot_extension_register_importer(godot_extension_importer importer). Thegodot_extension_importer` type would just be a struct with function pointers.

Loading and installing

A new directory called "extensions" containing subfolders for the different extension. In those folders a file "extension.cfg" or something like that. This file specifies the GDNativeLibrary to use to load the extension.

- extensions
    - fbx_loader
        - extension.cfg
        - fbx_library.tres
    - python_language
        - extension.cfg
        - python_library.tres

When the editor or game starts the extensions get loaded.

The Asset Lib could feature a new type of asset: "extension" and install that in the right place. Once installed then the extension would get initialized, editor restart should not be required. (the Asset Lib downloader needs to initialize the initialization process πŸ˜„ )

Use cases


WDYT?

reduz commented 7 years ago

I think this would be better as something specific to gdnative. We could call it "native modules" or something like that.

On Apr 24, 2017 2:27 AM, "Thomas Herzog" notifications@github.com wrote:

Extensions

After talking to @touilleMan https://github.com/touilleMan and @reduz https://github.com/reduz in Paris (it was awesome!!!) I'm writing down this proposal.

GDNative is a nice way to use third party libraries in your game or just optimize performance critical code, but it could also be used to add "extensions" to Godot (for the lack of a better name, "plugin" and "add-on" are already taken πŸ˜› ) Current situation

A GDNativeScript represents one class in a GDNativeLibrary. To have any effect, this script must be attached to any object. This is cool to write game code or wrap third party code in a class.

The current way to "extend" Godot is via plugins (or more precisely, "editor plugins"). These plugins can do a lot of awesome things, but they all only run in the editor. GDNative scripts can be used to make plugins as well, but they can't do anything that will have an impact on the running game. Proposal

[image: extension] https://cloud.githubusercontent.com/assets/5209613/25318699/7d9c0810-2894-11e7-865f-c4ceae5f6c84.png

Have a new project settings tab for "extensions". Extensions are GDNativeLibraries that don't need to expose any classes, they just get run and that's it.

These extensions get loaded at startup, in the editor as well as in a game. Implementation of an extension

A regular GDNativeLibrary, but it contains a special function godot_native_extension_init and godot_native_extension_terminate that get called at startup and shutdown of Godot.

The C API (modules/gdnative/godot.h and modules/gdnative/godot/) will contain new functions, like godot_extensionregister importer(godot_extension_importer importer). The godot_extension_importer type would just be a struct with function pointers. Loading and installing

A new directory called "extensions" containing subfolders for the different extension. In those folders a file "extension.cfg" or something like that. This file specifies the GDNativeLibrary to use to load the extension.

  • extensions
    • fbx_loader
      • extension.cfg
      • fbx_library.tres
    • python_language
      • extension.cfg
      • python_library.tres

When the editor or game starts the extensions get loaded.

The Asset Lib could feature a new type of asset: "extension" and install that in the right place. Once installed then the extension would get initialized, editor restart should not be required. (the Asset Lib downloader needs to initialize the initialization process πŸ˜„ ) Use cases

  • downloadable scripting languages in the AssetLib (with editor support)
  • new resource importers
  • possibly more that I can't think of right now πŸ˜„

WDYT?

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z23Oo0Lp0h6J8Z_LPd2tooQdF_biSks5ry-xogaJpZM4NFlw4 .

reduz commented 7 years ago

Maybe to make them totally clear and avoid confusion, they could be "gdnative submodules"

On Apr 24, 2017 8:13 AM, "Juan Linietsky" reduzio@gmail.com wrote:

I think this would be better as something specific to gdnative. We could call it "native modules" or something like that.

On Apr 24, 2017 2:27 AM, "Thomas Herzog" notifications@github.com wrote:

Extensions

After talking to @touilleMan https://github.com/touilleMan and @reduz https://github.com/reduz in Paris (it was awesome!!!) I'm writing down this proposal.

GDNative is a nice way to use third party libraries in your game or just optimize performance critical code, but it could also be used to add "extensions" to Godot (for the lack of a better name, "plugin" and "add-on" are already taken πŸ˜› ) Current situation

A GDNativeScript represents one class in a GDNativeLibrary. To have any effect, this script must be attached to any object. This is cool to write game code or wrap third party code in a class.

The current way to "extend" Godot is via plugins (or more precisely, "editor plugins"). These plugins can do a lot of awesome things, but they all only run in the editor. GDNative scripts can be used to make plugins as well, but they can't do anything that will have an impact on the running game. Proposal

[image: extension] https://cloud.githubusercontent.com/assets/5209613/25318699/7d9c0810-2894-11e7-865f-c4ceae5f6c84.png

Have a new project settings tab for "extensions". Extensions are GDNativeLibraries that don't need to expose any classes, they just get run and that's it.

These extensions get loaded at startup, in the editor as well as in a game. Implementation of an extension

A regular GDNativeLibrary, but it contains a special function godot_native_extension_init and godot_native_extension_terminate that get called at startup and shutdown of Godot.

The C API (modules/gdnative/godot.h and modules/gdnative/godot/) will contain new functions, like godot_extension_register_importer(godot_extension_importer importer). The godot_extension_importer type would just be a struct with function pointers. Loading and installing

A new directory called "extensions" containing subfolders for the different extension. In those folders a file "extension.cfg" or something like that. This file specifies the GDNativeLibrary to use to load the extension.

  • extensions
    • fbx_loader
      • extension.cfg
      • fbx_library.tres
    • python_language
      • extension.cfg
      • python_library.tres

When the editor or game starts the extensions get loaded.

The Asset Lib could feature a new type of asset: "extension" and install that in the right place. Once installed then the extension would get initialized, editor restart should not be required. (the Asset Lib downloader needs to initialize the initialization process πŸ˜„ ) Use cases

  • downloadable scripting languages in the AssetLib (with editor support)
  • new resource importers
  • possibly more that I can't think of right now πŸ˜„

WDYT?

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z23Oo0Lp0h6J8Z_LPd2tooQdF_biSks5ry-xogaJpZM4NFlw4 .

karroffel commented 7 years ago

Oh yeah, "GDNative submodes" sounds good!

reduz commented 7 years ago

Could be shortened to submodules too..

On Apr 24, 2017 9:15 AM, "Thomas Herzog" notifications@github.com wrote:

Oh yeah, "GDNative submodes" sounds good!

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296551528, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z24hu_l1ZyWEOvUcewZIXIFcxyOGPks5rzEwHgaJpZM4NFlw4 .

akien-mga commented 7 years ago

Or maybe change "Plugins" to "Script plugins" and add "Native plugins"? That would be the most explicit regarding what they are, and how they are used: both can be plugged into the engine, the former are script-based and interpreted, the latter are native code.

reduz commented 7 years ago

The problem is that plugins are editor plugins, while submodules are general purpose modules that extend engine support, and not really editor related.. so they are not quite the same

On Apr 24, 2017 9:52 AM, "RΓ©mi Verschelde" notifications@github.com wrote:

Or maybe change "Plugins" to "Script plugins" and add "Native plugins"? That would be the most explicit regarding what they are, and how they are used: both can be plugged into the engine, the former are script-based and interpreted, the latter are native code.

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296563064, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z21eYqdvTc-KWjgg1yROJfJyyeQosks5rzFSigaJpZM4NFlw4 .

reduz commented 7 years ago

As in, a plugin is on a higher level than a submodule, you could easily make a plugin that uses a submodule

On Apr 24, 2017 9:57 AM, "Juan Linietsky" reduzio@gmail.com wrote:

The problem is that plugins are editor plugins, while submodules are general purpose modules that extend engine support, and not really editor related.. so they are not quite the same

On Apr 24, 2017 9:52 AM, "RΓ©mi Verschelde" notifications@github.com wrote:

Or maybe change "Plugins" to "Script plugins" and add "Native plugins"? That would be the most explicit regarding what they are, and how they are used: both can be plugged into the engine, the former are script-based and interpreted, the latter are native code.

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296563064, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z21eYqdvTc-KWjgg1yROJfJyyeQosks5rzFSigaJpZM4NFlw4 .

reduz commented 7 years ago

Example of the above: A minecraft like world editor. You can have a submodule that generates the voxel terrain in C++. At the same time, an editor plugin which contains an UI to edit it, which is written jn gdscript. Both are part of a single addon that you get from asset lib.

So the relationship is like:

Addon -> Editor Plugin -> Submodule

On Apr 24, 2017 9:58 AM, "Juan Linietsky" reduzio@gmail.com wrote:

As in, a plugin is on a higher level than a submodule, you could easily make a plugin that uses a submodule

On Apr 24, 2017 9:57 AM, "Juan Linietsky" reduzio@gmail.com wrote:

The problem is that plugins are editor plugins, while submodules are general purpose modules that extend engine support, and not really editor related.. so they are not quite the same

On Apr 24, 2017 9:52 AM, "RΓ©mi Verschelde" notifications@github.com wrote:

Or maybe change "Plugins" to "Script plugins" and add "Native plugins"? That would be the most explicit regarding what they are, and how they are used: both can be plugged into the engine, the former are script-based and interpreted, the latter are native code.

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296563064, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z21eYqdvTc-KWjgg1yROJfJyyeQosks5rzFSigaJpZM4NFlw4 .

karroffel commented 7 years ago

good example! "Plugins" are actually "Editor plugins", "submodules" (not that sure about the name though, of what are they a sub-part?) are not specific to the editor.

The Minecraft example could make use of a submodule because those are loaded before editor plugins are loaded.

akien-mga commented 7 years ago

So in the end those are just pluggable "modules", as we understand the stuff in modules/. Users have already been confused by our choice of terms with modules vs plugins, so it might be a good opportunity to rethink that a bit for 3.0 :)

reduz commented 7 years ago

Yeah, you can still perfectly use gdnative for editor plugins.. this is a little lower level, so I suppose submodule conveys both that its a module based off another module.

The main source of confusion is imo the addons folder, which needs to be there because we can't install modules to custom locations... Which is something that we can probably do change in 3.0, as we deprecated xml and the remaining formats (scn and tscn) allow changing dependency location with little hassle. (This was next to impossible in xml).

So.. for 3.0 we could get rid of the addons folder altogether and let godot scan the plugins from EditorFilesystem normally.

With addons gone, we would be just left with EditorPlugin and SubModule (or better name).

Does this sound right?

On Apr 24, 2017 10:28 AM, "RΓ©mi Verschelde" notifications@github.com wrote:

So in the end those are just pluggable "modules", as we understand the stuff in modules/. Users have already been confused by our choice of terms with modules vs plugins, so it might be a good opportunity to rethink that a bit for 3.0 :)

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296574581, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z280RaPJ2c5CsX3CDJEr20g74PQS-ks5rzF0PgaJpZM4NFlw4 .

karroffel commented 7 years ago

sounds good to me!

I think we should choose the names carefully tough

Zylann commented 7 years ago

How is this different than having a regular GDNativeLibrary having an editor version and export version? Why does it need to be a specific thing?

You said plugins can't have an impact on the game, but it's wrong. My gdscript terrain plugin is able to run both in editor and in game. The only issue I had with it is about telling Godot not to export the editor-only part.

reduz commented 7 years ago

Editor just expects you to provide an EditorPlugin. For everything else it's the same.

On Apr 24, 2017 2:18 PM, "Marc" notifications@github.com wrote:

How is this different than having a regular GDNativeLibrary having an editor version and export version? Why does it need to be a specific thing?

You said plugins can't have an impact on the game, but it's wrong. My gdscript terrain plugin is able to run both in editor and in game. The only issue I had with it is about telling Godot not to export the editor-only part.

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296644080, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z26V6RoEoHVZU75KECFIm1C1jGRoPks5rzJMWgaJpZM4NFlw4 .

bojidar-bg commented 7 years ago

What's the issue of breaking compatibility with current plugins, and shipping the more-extensible "extensions"/"plugmodules"/"native plugins" with ability to add EditorPlugin nodes and to be made with GDScript?

That way, we would not be forced to use GDNative for simple importers (e.g. of a custom text-based format), while we can still do everything from the proposal. And, as an additional plus, one would be able to even use C# for import/export/stuff!

reduz commented 7 years ago

Breaking compatibility is only intended to install addons anywhere, and nothing else...

On Apr 24, 2017 3:30 PM, "Bojidar Marinov" notifications@github.com wrote:

What's the issue of breaking compatibility with current plugins, and shipping the more-extensible "extensions"/"plugmodules"/"native plugins" with ability to add EditorPlugin nodes and to be made with GDScript?

That way, we would not be forced to use GDNative for simple importers (e.g. of a custom text-based format), while we can still do everything from the proposal. And, as an additional plus, one would be able to even use C# for import/export/stuff!

β€” You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/issues/8505#issuecomment-296668834, or mute the thread https://github.com/notifications/unsubscribe-auth/AF-Z2yt9x2P-3SDcZoYfDOMMZbBjs_73ks5rzKP-gaJpZM4NFlw4 .

Zylann commented 7 years ago

Installing addons anywhere is already a separate issue though https://github.com/godotengine/godot/issues/6023

RameshRavone commented 7 years ago

Just wanted to know the status (is it being developed or dropped)

touilleMan commented 7 years ago

@RameshRavone now that my Python binding is working, I'm going to start working on this feature

no promise on the eta though ;-)

karroffel commented 7 years ago

@touilleMan I started laying down the basics for that on my fork, if you want you could help develop it there. I wrote you a PM on Discord about that too

reduz commented 7 years ago

how is this going?

touilleMan commented 7 years ago

@reduz pretty well ^^

You can have a look on my fork and the corresponding Python pluginscript module.

So far I got my Pong example written in Python working :smile: I still have to bind the editor and debugger functions though.

karroffel commented 7 years ago

This gets partially implemented by #10921, there's basically only a proper UI missing and then more APIs to actually use (like pluginscript)

HummusSamurai commented 7 years ago

Closed by https://github.com/godotengine/godot/pull/11953 ?

karroffel commented 7 years ago

Well it's more like an ongoing effort, PluginScript is only one application, GDNative singletons basically solve this issue.

But you are right, it can be closed now.