search

godotengine / godot-docs

Godot Engine official documentation
https://docs.godotengine.org
Other
3.78k stars 3.07k forks source link

Create a complete and concise documentation for GDExtension #5618

Open nonunknown opened 2 years ago

nonunknown commented 2 years ago

Describe the project you are working on

A G.O.A.P Implementation for Godotengine via GDExtension

Describe the problem or limitation you are having in your project

Me and a lot of users has little to none knowledge about GDNative and C++, now that GDNative is "deprecated" in 4.x we have even less knowledge about such powerful tool, some of them:

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The documentation will have a "Zero to Hero" subject, where a user with basic knowledge of C++ will learn:

I may have forgot some stuff, so if you're reading this, please comment below suggestions

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

A Complete documentation in the docs page

If this enhancement will not be used often, can it be worked around with a few lines of script?

Nope, its a documentation.

Is there a reason why this should be core and not an add-on in the asset library?

this cant be addon, since is GDExtension.

Gnumaru commented 2 years ago

Shouldn't this belong to the other project, godot docs?

https://github.com/godotengine/godot-docs/

nonunknown commented 2 years ago

not sure, there is for bugs and enhacements related to existing documentation, @Calinou can confirm?

Xrayez commented 2 years ago

GDExtension appears to be still work-in-progress. But GDExtension without even preliminary documentation/specification is going to be quite useless, by useless I mean this may prevent adoption by potential developers that would like to improve and build upon GDExtension (like me), and those people who could literally speed up the development. I think GDExtension should get more love and support, so I'd leave this proposal here... 😛

I see no technical specification for how GDExtension should work or supposed to work, not even talking about user documentation. So it's more about lack of specification as I see it.

This could potentially be moved to https://github.com/godotengine/godot-proposals/discussions.

See also proposal of mine:

nonunknown commented 2 years ago

Yep GDExtension is WIP, which means the best time for thinking about how to present the documentation for end-users is now! Also I dont think GDExtension is "useless" it makes life easier when you need to deal with more advanced stuff, having the benefits of the front-end (godot editor) and back-end (core source code).

Xrayez commented 2 years ago

I dont think GDExtension is "useless" it makes life easier when you need to deal with more advanced stuff, having the benefits of the front-end (godot editor) and back-end (core source code).

See my definition of "useless". If the feature is not documented - nobody knows about it, therefore it cannot be used by most users who are not original developers, hence useless, I should've put "useless" in quotes I guess.

nonunknown commented 2 years ago

yep I got it, but your arfimation "feature is not documented - nobody knows about it" is not true, I know little about C++, also I had little knowledge about GDNative, and even with that I was able to get more or less how GDExtension works, I cant even understand its source code, neither from godot engine. Its all about dedication in the end of the day!

PS: the "example" folder which comes with godot-cpp explain very well how to do basic and advanced stuff.

azur-wolf commented 2 years ago

Who is the developer/contributor responsible of GDExtension?

For now idk what GDExtension will be a part from the GDNative substitution. Its an absolute stranger.

@Xrayez mentioned that GDExtensions could throw some issues for addon/plugin developers (recompilation or something iirc).

So its true that its better discuss GDExtension now before its completion, or we could regret it later..

PS: the "example" folder which comes with godot-cpp explain very well how to do basic and advanced stuff.

Do you mean the "test" folder?

Xrayez commented 2 years ago

@Xrayez mentioned that GDExtensions could throw some issues for addon/plugin developers (recompilation or something iirc).

The difference between GDNative and GDExtension is not substantial in principle. But if GDExtension aims to replicate and/or replace C++ modules development as completely as possible, might as well replicate everything that makes C++ modules development a breeze currently.

Who is the developer/contributor responsible of GDExtension?

I think this is mostly a work by @reduz and @vnen as of now.

azur-wolf commented 2 years ago

Then I understood incorrectly (it was from your discussion to being able to write plugins integrated with the editor on GDScript).

Not sure what was more problematic, if GDNative or Modules. Hopefully GDExtension will solve most of that.

Xrayez commented 2 years ago

Not sure what was more problematic, if GDNative or Modules. Hopefully GDExtension will solve most of that.

Both are equally problematic to be honest, depending on who you ask (user vs developer). Those are actually two different technologies I'd say. See discussion in godotengine/godot-proposals#565 if you really want to delve into this topic more. Ultimately, I think this boils down what kind of approach will be supported the most.

azur-wolf commented 2 years ago

Now I get it:

GDNative → share without re-compile engine, ideal for addons/plugins.

Modules → need re-compile engine, so better if have a high-tier Threadripper, but has not the vulnerabilities of GDNative.

Excuse me for asking too much, don't want to bother; There are additional problems with Godot Modules?

Xrayez commented 2 years ago
  • can be compiled as dynamic library and loaded against the project binary (also can against the Editor?)

There's some distinction between dynamic/shared libraries with modules, which are in theory doable but those have issues as well (Godot in particular):

Zireael07 commented 2 years ago

I still fail to understand how GDExtension works (and the comment above mentioning GDNative has me confused because wasn't GDExtension supposed to be a replacement for GDNative)...

nonunknown commented 2 years ago

@Zireael07 GDExtension is something like GDNative 2.0, the difference is that GDExtension has access to the core of the engine in a "deeper level" (has access to more stuff).

basically GDnative and GDExtension is a compiled DLL,SO,DYLIB which communicates with the engine API, so you can use it to make an entire game like gdscript or only for specific stuff like integrate an new audio engine without recompiling the entire engine!

azur-wolf commented 2 years ago

There are any news/progress regarding GDExtension? Who is the maintainer?

nonunknown commented 2 years ago

here is not place for this, but you can find fresh news here: https://chat.godotengine.org/channel/gdextension

venilark commented 2 years ago

Is GDExtension what I should use to replicate a single GDScript/Node file in C++ (for better performance) or is it for something completely different? From @nonunknown's answer I think it should but confirmation would be appreciated, for a couple of calculation-heavy scripts C++ would be the way to go, only if this new "C++'d Node" can communicate with other nodes written in GDScript

Calinou commented 2 years ago

@venilark Yes, this is exactly what GDExtension is about :slightly_smiling_face:

While a single GDExtension is generally used to power multiple classes, it can also be used with a one-library-per-class approach.

venilark commented 2 years ago

@venilark Yes, this is exactly what GDExtension is about 🙂

While a single GDExtension is generally used to power multiple classes, it can also be used with a one-library-per-class approach.

thank you 🙂

venilark commented 2 years ago

@venilark Yes, this is exactly what GDExtension is about 🙂

While a single GDExtension is generally used to power multiple classes, it can also be used with a one-library-per-class approach.

Another question crossed my mind. If I write a function that returns a Dictionary or any object in the C++ node or GDScript node, can the other one call it and get its result? what if it's a dictionary made of different objects, could this be a problem in C++?

nathanfranke commented 2 years ago

@venilark Yes, this is exactly what GDExtension is about slightly_smiling_face While a single GDExtension is generally used to power multiple classes, it can also be used with a one-library-per-class approach.

Another question crossed my mind. If I write a function that returns a Dictionary or any object in the C++ node or GDScript node, can the other one call it and get its result? what if it's a dictionary made of different objects, could this be a problem in C++?

(IIRC) In order to access C++ methods from GDScript, they need to be registered with the ClassDB. If the function returns or has parameters of any type other than a variant (e.g. floats, Dictionaries, Nodes, Resources, etc.) it will give an error.

Zireael07 commented 2 years ago

I find myself needing to rework some stuff in my projects to be better/faster. GDExtension is still missing from the docs so many things are still unclear. E.g. can I call a GDE function/use a GDE data structure from GDScript?

Does it only support C++ now? I'm given to understand c# might be migrating to GDE soon(ish).

azur-wolf commented 2 years ago

I find myself needing to rework some stuff in my projects to be better/faster. GDExtension is still missing from the docs so many things are still unclear. E.g. can I call a GDE function/use a GDE data structure from GDScript?

Does it only support C++ now? I'm given to understand c# might be migrating to GDE soon(ish).

IIRC the implementation of GDExtension was facing certain design problems. Right now dont know which is its state. (I assume that you're using Godot 4 alpha)

the only way to clear those doubts is to ask directly to the devs on RocketChat

benbot commented 2 years ago

WIll the GDExtension api be stable by 4.0 beta?

If so it'd be a great time to start writing documentation! I'd be happy to contribute if at all possible :)

paddy-exe commented 1 year ago

Bump! Hey, so I started already with docs for the GDExtension system. I would be incredibly grateful if anyone wants to help since all the renames (GDNative to GDExtension) themselves are a ton of work. I will mainly focus on C++ first so if anyone wants to have a go at the C example, that would be great. @benbot tagging you since you showed interest

Frontrider commented 1 year ago

@paddy-exe how is it coming along?

paddy-exe commented 1 year ago

@paddy-exe how is it coming along?

I worked out all the feedback in my PR. I need a contributor looking over it. Also someone who knows the underlying system needs to confirm that the information I provided is correct.

arocull commented 1 year ago

Not sure where to ask this, but is there any documentation on the "[dependencies]" portion of .gdextension files? I can't seem to find any info, and the only extensions I can find are tutorial ones which don't require external libraries.

paddy-exe commented 1 year ago

Not sure where to ask this, but is there any documentation on the "[dependencies]" portion of .gdextension files? I can't seem to find any info, and the only extensions I can find are tutorial ones which don't require external libraries.

There are only public GDExtensions where you can see the use. It's not yet in the docs.

allan-simon commented 1 year ago

There are only public GDExtensions where you can see the use. It's not yet in the docs.

However the code inside gdextension.cpp does not seem to use it ?

Do you happen to have a link of a github using it ?

Zireael07 commented 1 year ago

Bump. 4.1 is close and there still seem to be things missing from the docs...

EDIT: For instance, "C++ modules provide deeper integration into the engine. GDExtension's access is not as deep as static modules" is very much unclear. What does "not as deep" access means in practice? What are things modules CAN do and GDExtension can't?

nonunknown commented 1 year ago

Bump. 4.1 is close and there still seem to be things missing from the docs...

EDIT: For instance, "C++ modules provide deeper integration into the engine. GDExtension's access is not as deep as static modules" is very much unclear. What does "not as deep" access means in practice? What are things modules CAN do and GDExtension can't?

If I could tell one thing, is debugging, you cant debug GDExtension as far as I know

vnen commented 1 year ago

If I could tell one thing, is debugging, you cant debug GDExtension as far as I know

You can debug GDExtension. Might not be as trivial as debugging a module. You should use a debug build of both Godot and the GDExtension, but it should just work.

For instance, "C++ modules provide deeper integration into the engine. GDExtension's access is not as deep as static modules" is very much unclear. What does "not as deep" access means in practice? What are things modules CAN do and GDExtension can't?

What happens is that a module can access anything in Godot's codebase, since it's being compiled together. GDExtension can only access what has been exposed to the API. This sense definitely needs some elaboration, especially because for the vast majority of applications this difference is not relevant (you rarely ever need to access non-exposed stuff and if you do is likely that it should be exposed, which would be the better approach).

We still lack someone who is knowledgeable in GDExtension land and is willing to write this up and has the time to do so.

dsnopek commented 1 year ago

You can debug GDExtension. Might not be as trivial as debugging a module. You should use a debug build of both Godot and the GDExtension, but it should just work.

I can confirm this! In working on SG Physics 2D, I'm able to run the debugger and set breakpoints in both my GDExtension and Godot itself.

nonunknown commented 1 year ago

ohh that's great news then :D

Zireael07 commented 1 year ago

One more thing - the docs don't make it clear whether GDExtension works on ALL platforms Godot exports to, or just some?

(Usual suspects are web and Android)

Frontrider commented 1 year ago

One more thing - the docs don't make it clear whether GDExtension works on ALL platforms Godot exports to, or just some?

(Usual suspects are web and Android)

You need a binary built for that platform, and the gdextension file provides fields for those.

paddy-exe commented 1 year ago

One more thing - the docs don't make it clear whether GDExtension works on ALL platforms Godot exports to, or just some?

Android should work IIRC and Web export currently doesn't work due to a dependancy issue with emscript

dsnopek commented 1 year ago

I think there may be limitations with regard to iOS support as well, due to some aspect of how the platform works? I'm not 100% sure about that, though.

Also, I think it should be possible to get web working eventually - someone just needs to do the work!

the-ssd commented 1 year ago

Let's say, for example, I am trying to create bindings for a compiled language. How do I do that? Example project in C will help A LOT

nonunknown commented 1 year ago

good Idea, was telling vnen this some days ago! I'll add to the list above!

the-ssd commented 1 year ago

Example usage of GDExtension in the C language

maybe change to Example usage of Low Level GDExtension in the C language

Frontrider commented 1 year ago

A detailed description on what each method in the header does, with examples? (possibly including code from the c# binding as well as another example?)

nonunknown commented 1 year ago

@Frontrider can you be more detailed, didnt understood!

Frontrider commented 1 year ago

@nonunknown What I mean, is to have a very (if not overly) detailed description for each method in the generated header. If you think something should be natural, at least include a reference to external documentation anyways. This should make it as accessible as humanly possible.

Same for the generated json, a schema (to generate parsers, the way it is set up now basically needs to be handled by hand all the way through, even knowing that we have hundreds of tools specifically for this), and an explanation should be there.

I know that the c# binding is a module, and not an extension at the moment but that one is also a good "advanced example" as well, as it binds to a language with its own runtime.

MoralCode commented 10 months ago

Not sure where to ask this, but is there any documentation on the "[dependencies]" portion of .gdextension files? I can't seem to find any info, and the only extensions I can find are tutorial ones which don't require external libraries.

Ive been looking for this too. Since there seems to be a lack of examples, here's a link to my project which - once I figure it out - should contain an example of a GDExtension file with a dependencies section.

The source code at https://github.com/godotengine/godot/blob/3e7f638d7b574785f521beafaf52a6ad95be016f/editor/plugins/gdextension_export_plugin.h#L127 seems to suggest that it parses the keys of the dependencies section to determine what to load as a shared object, but it seems like this is not actually loading them or something else is happening (also this is export code, not import code and I can't find documentation or tutorials on if/how GDExtensions can be created in the Godot GUI)

allan-simon commented 10 months ago

@MoralCode you can check my extension here https://github.com/allan-simon/gdextension-vosk-speech-to-text

as I had finally figured out how dependencies was working https://github.com/allan-simon/gdextension-vosk-speech-to-text#interesting-part

dsnopek commented 10 months ago

seems to suggest that it parses the keys of the dependencies section to determine what to load as a shared object, but it seems like this is not actually loading them

To clarify here (although this is explained on Allan Simon's link too), the [dependencies] section only tells Godot that those files should also be exported. Actually loading any shared objects to use them is up to the GDExtension itself to do (which could be as simple as telling the dynamic linker about them when building your GDExtension, ie -l with GCC).

leandro-benedet-garcia commented 10 months ago

Sphinx is able to generate a documentation from a file, the file in https://github.com/godotengine/godot/blob/master/core/extension/gdextension_interface.h has a lot of documentation in the file that in my opinion would be nice to have included into the sphinx docs.

However, I noticed a issue, the documentation is a different project than the source code, to automatically generate the docs we would need to pull the source code, or pull specific files. But I still believe it would be worth it because there is a bunch of material already there, and kinda hidden away.

Using the docs that we already have would be interesting in my oppinion

vnen commented 10 months ago

However, I noticed a issue, the documentation is a different project than the source code, to automatically generate the docs we would need to pull the source code, or pull specific files.

The class reference is also in the main source repository and not here, so I don't think it's more complex as long as there are a simple tool to do it. The contents can be synced manually when needed.

I do agree that generating from the header is better if it's an option, since we can have a single source while keeping the documentation in the source file itself.