godotengine / godot-proposals

Godot Improvement Proposals (GIPs)
MIT License
1.06k stars 65 forks source link

Godot C# GDExtension Roadmap Discussion #7895

Open reduz opened 8 months ago

reduz commented 8 months ago

NOTE: This proposal aims to be a public technical discussion regarding the future of C# in Godot. The idea is to get help from the community to reach implementation direction consensus.

Describe the project you are working on

Godot C# Bindings

Describe the problem or limitation you are having in your project

The current implementatio of C# in Godot is done on top of the script layer directly and in C++ using the .NET hosting API. This is due to how Godot worked in version 3. It currently has the following problems:

Describe the feature / enhancement and how it helps to overcome the problem or limitation

The proposed solution is to move C# support to the new extension system in Godot 4.0. This will give C# in Godot effective first class support at the same level C++ has, bringing the following features:

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

NOTE This is a medium term goal for C#. Currently, our immediate priorities are fixing C# support on mobile and web. What is described in this proposal is work that would happen in parallel.

Ideally, we want to move Godot to use the GDExtension API instead of the current custom glue glue. Unlike GDExtension, though, we don't want to depend on a glue API, it should be possible to open the .NET DLL containing the Godot C# library and have it register itself via the GDExtension API through a tiny boostrapper that detects if a .net runtime is installed.

This would make the entire C++ side far simpler, easier to maintain, and easier to implement new features (such as many additions on the GDExtension side currently are not supported in in the current module, such as GDVIRTUAL, struct pointers, Godot collections, enhanced performance due to less glue, etc).

Additionally, it would make it easy to have a single Godot executable and export templates, where C# can be plugged at run-time easily.

Here are the steps to evaluate that would be required to do this:

Creation of a Godot Library in C

The main idea is that C# should act as if it was a C++ extension. A library (in the shape of a .net DLL), probably GodotLibrary.dll, can be added to a project and this automatically is recognized and opened by Godot as if it was a native shared object, but done actually via the .NET runtime.

In other words, if the C# runtime is installed in the Godot project (or supplied by the platform to Godot), it is used to open GodotLibrary.dll and call an entry point that should be the same as a regular GDExtension,

Rewrite the extension interface in C

Godot has a C extension interface defined in C. This extension interface should be available as part of the GodotLibrary.dll C# source code. Probably something like ExtensionInterface.cs.

This means making all the structs and C funtions available in unsafe mode as their C# counterparts.

Binder generation

Once we have the extension interface defined in C#, we can start working on the binder generator. Before going into this, I think it's important to reach consensus on whether we want to generate the bindings as scripts or extension.

Bindings as extension or script?

Godot has two possible workflows today and both are supported in GDExtension: Script Workflow and Class Extension Wokflow.

Normally, script workflow is very quick and intuitive for GDScripts, as the steps to create a new script are:

Blazingly fast. In contrast, for C# the script workflow is more cumbersome:

The alternative is to implement C# using the Class Extension workflow. This workflow is the one compiled languages like GodotCPP, Rust, Swift, etc. uses.

In C#, it would work like this:

As you can see, when using compiled languages, there are not significant differences.

The question is, should C# using script or extension class bindings?

Here are the pros and const:

Script

Pros:

Cons:

Extension class

Pros:

Cons:

Should we use Script or Class extension workflow? That is up to you community.

My personal recommendation would be to go with Class Extension workflow primarily due to performance reasons. With this workflow you will be able to always have optimal performance in all situations and the alternations to ease of use are very minimal.

EDIT: I added a small poll a few coments below.

I put together a quick FAQ:

**Q**: Will I be able to reload extensions like scripts?
**A**: Yes

**Q**: If when loading a project, some extension class is missing, is this 
       lost upon scene load?
**A**: No, extensions create placeholders so you can re-save your scene and open 
       it again when the class exists.

**Q**: Can both workflows be provided?
**A**: There is some discussion on the GodotCPP side with this same question. 
       Its hard the same class is provided as both script an extension, so one 
       possibility is to have something like `CSharpScript<BaseClass>` you inherit, 
       then you access the actual engine class via a "base" property, example you 
       change position by doing `base.position = Vector2(20,30)`. This would allow both workflows.

**Q**: Will I have to redo my C# project if we go with Extension Class workflow?
**A**: Minimal changes will be needed and a converter for scenes can be provided 
       so the work you need to do is minimal.

New Bindings Generator

The new bindings generator would comprise of the following parts:

ExtensionInterface

As mentioned before, the extension_interace file from Godot rewritten in C# as ExtensionInterface.cs.

Everything else will use this and it replaces the glue/ code in the existing modules/mono codebase.

Glue

The C# side of the glue that previously existed in mono/glue, which will need to be used by both Godot Types and the bindings generator. This talks to ExtensionInterface.cs to ensure that types such as Callable, Signal and a few low level behaviors in the class bindings can be implemented properly.

Godot Types

These are the basic Godot types (Vector3, Vector4, Transform2D, etc) exposed to C#. They are currently located in this directory. As mentioned before, not all need to be written natively, some now can be done 100% by using the binding generator.

The Godot collections (the way C# calls Godot PackedArrays, Array, Dictionary) currently are not properly implemented in C#, C# always does conversion from native C# types to Godot types on every call. Ideally this should be kept because it makes the API easier to use, but all the Godot array types and dictionary should be properly supported as Godot.Collections. This will allow properly implementing proposal

7842.

Additionally, as mentioned, some may need minor changes, such as Callable or Signal, which now need to use ExtensionInterface.cs.

Bindings Generator

This is the largest part of the work. The new bindings generator can be written in Python, C# or whathever decided best. It takes an extension_api.json file and dumps the whole Godot API as C#.

All the classes, utility functions (currently hardcoded), and several other binding information is used to create the Godot API in C# entirely in procedural form. Again, no C++ glue code should be needed anymore. Everything accessed via ExtensionInterface.cs.

Additionally, this should use the compatibility hashes system in GDExtension, so we can ensure that backward compatibility is also kept in C# when Godot releases new versions.

Currently, Godot C# uses API hashes, again those need to be dropped entirely and API hash support removed from Godot.

Editor Tools

Finally, porting the existing editor tooling to the new system. Most code probably will remain unmodified.

Boostrapper

A small piece of code to load the .net runtime (if installed by the user), that calls the C# entry point.

Assembling the GodotLibrary.dll

The GodotLibrary.dll will be built from the existing and generated C# files created by the bindings generator script.

On PC platforms, It will be shipped in Godot together with:

Compiling the game

C# games in Godot would simply be a library compiled against GodotLibrary.dll and other .net dependencies. This step is the same as now.

Deploying to platforms

An important question here is how will this be ported to different platforms (iOS, C#, etc).

Essentially, nothing should really change. Platforms that run JIT will ship with the runtime.

Platforms that use NativeAOT will need to just change how the entry point is accessed, but all the work is now done inside C#, so all Godot cares is reaching that entry point and that's it.

Closing words

Thanks for reading the C# roadmap so far. Feel free to discuss what you think in this proposal!

If this enhancement will not be used often, can it be worked around with a few lines of script?

N/A

Is there a reason why this should be core and not an add-on in the asset library?

The tiny boostrapper will be core, the rest obtainable separately.

paulloz commented 8 months ago

Since I don't see it mentioned anywhere, I just want to paste below the minutes from last week's .NET contributors meetings regarding this specific topic. Note that I'm just quoting, I don't personally share the same opinion on all of the points.

Editor unification (.NET integration as a plugin/extension) (switch to integration as gdextension or just always bundle it)

  • Good way to move forward. Decouple C# support from the core. If anything is missing in GDExtension, it would be missing for other languages too.
  • Would need to drop the entire interop code and rewrite it from scratch. Module uses a number of APIs which might not be exposed to GDExtension yet.
  • C++ extension would not needed to be reloaded, and it would be able to reload C# assemblies just like the module does currently.
  • Performance difference need to be assessed - attendees not convinced this would make a difference to the current approach.
Visne commented 8 months ago

rodmap

Edit: apologies for the joke, it's really great to see this problem being taken seriously πŸ˜„

reduz commented 8 months ago

@Visne

rodmap

Note to self: Do not type on the PC without the glasses.

JoNax97 commented 8 months ago

For my current/future use case (using C# to heavily modify Godot itself, basically creating a new editor), the extension class workflow would be hugely beneficial, if not outright necessary.

reduz commented 8 months ago

Quick poll!

If you favor script workflow (slightly better usability, but more limited in performance and extensibility), give this comment a :rocket: If you favor class extension workflow (better performance, ability to override engine internals, simpler code to maintain): give this comment a :heart:

Zylann commented 8 months ago

Some questions:

LauraWebdev commented 8 months ago

I just want to weight in that the current dev experience is pretty comfortable. I usually have my C# IDE on my second screen with the Godot Editor opened from the IDE.

There is one odd behavior where the Godot Editor sometimes seems to reset a source file (which I can revert easily) but I'm not entirely sure if that's a problem connected with the general structure.

One thing I'd like to know is how dependencies would be resolved in a user-scenario? Let's say I want to publish a tool written in C#, how would the user experience (of those who download said tool) be affected?

moonheart08 commented 8 months ago

Just going to weigh in that ideally whichever workflow is chosen should avoid Variant/Godot object overhead. I've seen the whole discourse over GC recently and think it was missing the forest for the woods that is having to interface with the engine using an indirect reflection-like system instead of direct function calls.

I really think that forcing non-scripting languages to go through a dynamic API is a bad idea in general, strongly typed languages have strong typing for a reason and it should be utilized at every level of the API (including at the FFI point)

Thus, I'd advocate for the use of extensions here.

c-schembri commented 8 months ago

My opinion is that if you're using C# in the first place, you're only worrying about performance and maybe third party libraries. I'd wager most C# developers care more about the former. On that point, whatever allows greater performance (or enables code which can facilitate greater performance), should be the option chosen.

Developer usability should be prioritised behind this. If we want nice usability, we can just use GDScript.

moonheart08 commented 8 months ago

My opinion is that if you're using C# in the first place, you're only worrying about performance and maybe third party libraries. I'd wager most C# developers care more about the former. On that point, whatever allows greater performance (or enables code which can facilitate greater performance), should be the option chosen.

Developer usability should be prioritised behind this. If we want nice usability, we can just use GDScript.

Hard disagree here, GDScript is unsuitable for large scale/modular/moddable projects due to lackluster language design (lack of interfaces and composition tools, lack of reliable typing i.e. you can fall back into pure dynamic types pretty easily and no longer get error checking, lack of compile time type verification, etc), but that's another ballpark for another day.

reduz commented 8 months ago

@Zylann Exposing a [Tool] interface is probably the same, a placeholder class is created and used during editor run.

Classes from C++ extension should become available easy if the headers are generated for them, but there is not an easy way to do this yet. Its not technically complex, just that some UI/UX needs to be designed.

JoNax97 commented 8 months ago

if you're using C# in the first place, you're only worrying about performance and maybe third party libraries. (...)

Hard disagree here, GDScript is unsuitable for large scale/modular/moddable projects due to lackluster language design (...)

While I don't agree GDScript is badly designed (I believe it seeks different design goals), I think it's true and important to highlight that modularity, scalability and refactorability are important aspects for choosing C#.

SamPruden commented 8 months ago

This is really exciting! I just have a few clarifying questions:


This means making all the structs and C funtions available in unsafe mode as their C# counterparts.

Am I understanding correctly that the plan here would be to expose low level APIs directly to C# (in an automated way based on GDE) but that high level APIs would be built on top of this (semi automated) for good user experience?

I would generally support that approach. Allowing each language to tailor the API design to its idioms rather than directly exposing GDE seems very important.


Is it in scope to make modifications to how the core APIs and GDE work to better support this, for C# and for other languages?

For example, is it reasonable to think about defining the core APIs (as registered in ClassDB) to work with very general purpose collection types (maybe even iterator like constructs for array inputs) and for languages to have the freedom to bind to those APIs using whatever wrapper is appropriate?

More about the motivation for that is here in my comment on the other thread.


To what extent would the plan be to try to retain backcompat with existing C# APIs, and to what extent can this be treated as a blank slate? Would it be reasonable to take a blank slate approach and ship both versions side by side for a while (perhaps billed as Stable and Experimental) so the user can choose one or the other on a per project basis, allowing the freedom to make breaking changes? (A cool feature of shipping a new version as Experimental is that breaking changes could continue in minor releases until it gets upgraded to Stable, giving more freedom to respond to feedback over a longer time period.)

Zylann commented 8 months ago

Classes from C++ extension should become available easy if the headers are generated for them

I mainly asked about the C# side having access to the C++ classes, I suspect C++ depending on C# is way less common in practice.

dsnopek commented 8 months ago

@Zylann:

  • Extension classes work as if they were core classes. One of the main behavior differences is that they always run, i.e there is no extra concept of optional @tool or [Tool] (whatever this is in C#), the classes will be really instantiated and code will run in the editor when added to the scene, as a side-effect to be integrated in a more low-level manner. Will it be the way it will work, or will [Tool] be added to C# extensions? (which means ClassDB I guess)

This is something we should discuss in general, outside of the context of C#. Now that we have a solution for hot reload, the next thing down the list that people using GDExtension for gameplay code complain about is "everything runs like a tool script!?" :-) We need an answer to this for C++ and all the other languages as well

rodrigofbm commented 8 months ago

My opinion is that if you're using C# in the first place, you're only worrying about performance and maybe third party libraries.

This kind of comment is unnecessary. You're just giving assumption what others people cares about. Focus on the thread or speak for yourself.

andy-noisyduck commented 8 months ago

@reduz

Quick poll!

If you favor script workflow (slightly better usability, but more limited in performance and extensibility), give this comment a πŸš€ If you favor class extension workflow (better performance, ability to override engine internals, simpler code to maintain): give this comment a ❀️

A poll given to a tiny subset of Godot's total C# userbase, framed with reasonably biased context from the engine lead? C'mon, this is not useful data. We can surely do better than this.

@c-schembri

My opinion is that if you're using C# in the first place, you're only worrying about performance and maybe third party libraries. I'd wager most C# developers care more about the former. On that point, whatever allows greater performance (or enables code which can facilitate greater performance), should be the option chosen.

Developer usability should be prioritised behind this. If we want nice usability, we can just use GDScript.

Let's not go making massive sweeping statements. There are many reasons people want to use C#. It indeed has better library support as you have mentioned. Tooling is more mature - be that the available editors, CI/CD environments, testing tools, build tools, 3rd party developer support etc.

Many developers choose C# because they have an existing pool of knowledge they use - whether that's their own existing knowledge a solo dev, or for studios where employees are more likely to have existing C# experience. They might want it as the path of least resistance to convert from Unity, something we've seen a lot of the last few weeks.

Outside of a few heavy algorithmic cases (proc gen etc), the engine should be doing all the heavy lifting and the performance of the scripting language is immaterial. The suggestion to make something less usable for everyone in order to benefit those cases seems off the mark.

c-schembri commented 8 months ago

@reduz

Quick poll! If you favor script workflow (slightly better usability, but more limited in performance and extensibility), give this comment a πŸš€ If you favor class extension workflow (better performance, ability to override engine internals, simpler code to maintain): give this comment a ❀️

A poll given to a tiny subset of Godot's total C# userbase, framed with reasonably biased context from the engine lead? C'mon, this is not useful data. We can surely do better than this.

@c-schembri

My opinion is that if you're using C# in the first place, you're only worrying about performance and maybe third party libraries. I'd wager most C# developers care more about the former. On that point, whatever allows greater performance (or enables code which can facilitate greater performance), should be the option chosen. Developer usability should be prioritised behind this. If we want nice usability, we can just use GDScript.

Let's not go making massive sweeping statements. There are many reasons people want to use C#. It indeed has better library support as you have mentioned. Tooling is more mature - be that the available editors, CI/CD environments, testing tools, build tools, 3rd party developer support etc.

Many developers choose C# because they have an existing pool of knowledge they use - whether that's their own existing knowledge a solo dev, or for studios where employees are more likely to have existing C# experience. They might want it as the path of least resistance to convert from Unity, something we've seen a lot of the last few weeks.

Outside of a few heavy algorithmic cases (proc gen etc), the engine should be doing all the heavy lifting and the performance of the scripting language is immaterial. The suggestion to make something less usable for everyone in order to benefit those cases seems off the mark.

The performance of the scripting language is not immaterial. Such a statement does not even make sense.

reduz commented 8 months ago

@dsnopek

This is something we should discuss in general, outside of the context of C#. Now that we have a solution for hot reload, the next thing down the list that people using GDExtension for gameplay code complain about is "everything runs like a tool script!?" :-) We need an answer to this for C++ and all the other languages as well

In C# this is not so terrible because you can sort of expose the property metadata (type, hints, etc) with a tag, which you can parse from C# itself before instantiating the class, then when instantiating in the editor a placeholder class is supplied that lets you update the properties.

In GodotCPP this is kinda a bit harder, you have this information from _bind_methods, so /at least/ this has to be registered. The problem is that when we instantiate a tool class we have to also give a separate placeholder instance that edits those properties.

IMO most of this can be solved from the Godot side without that much issue, but will require scratching our heads a bit.

definitelyokay commented 8 months ago

I would be okay with moving to a GDExtension implementation if (and ONLY if):

I would still prefer a C# entry-point where Godot is used as a library over either approach, but I doubt that would be a popular choice since it would probably take a lot of effort. I also recognize that C# can't be the entry point on all platforms (i.e., consoles), so there would have to be a second type of integration, anyways.

vikram3 commented 8 months ago

I want to use GD script, can gdscript converted into c# code πŸ˜Άβ€πŸŒ«οΈ

mahdisml commented 8 months ago

Does this idea or proposal make a difference in the output with Native AOT in dotnet 8?

andy-noisyduck commented 8 months ago

The performance of the scripting language is not immaterial. Such a statement does not even make sense.

Don't crop off the half the context and complain it makes no sense. It was specifically framed with "Outside of a few heavy algorithmic cases...".

The scripting language is not doing the heavy lifting in most cases - the engine is. Does it matter if your scripting language is x% quicker if the engine is doing an order of magnitude more work? For most people the answer should be no. It's the same reason there isn't a simple answer when people ask "will my Godot game be faster if I make it in C#?" - it depends. You're arguing to make the experience worse (for everyone) to benefit a small subset of use cases.

It's not even all algorithmic cases that benefit either. Its algorithmic operations that do reasonably heavy engine-interop. I'm sure that will benefit some developers. I remain unconvinced however that it will benefit most developers, and I remain less convinced by the argument that it's OK to trade developer usability in exchange for this gain.

reduz commented 8 months ago

@definitelyokay

I would be okay with moving to a GDExtension implementation if (and ONLY if):

Just to clarify, the current binder code and GDExtension are redundant and the same thing. The main advantage of this is that it costs much less to keep C# up to date for contributors.

I would still prefer a C# entry-point where Godot is used as a library over either approach

This was discussed, but ultimately rejected because Godot is not a C# focused engine. There is some ongoing discussions on allowing Godot to be compiled like a library, so it may be usable like this, but using it this way (say you get Godot itself on NuGet) would not be endorsed officially.

reduz commented 8 months ago

@andy-noisyduck I understand the point you are trying to make, but a large majority of users and contributors want a performance focused C# integration. Its a matter of preference that dictated the priority, from there we work on making the usability as good as possible.

JoNax97 commented 8 months ago

I agree that the poll may be biased. Most people seeing this proposal are probably the ones that followed last weeks drama and are interested in performance, after all.

nockawa commented 8 months ago

To me this is not script versus class extension, both are good, they are targeting different usage. But I wouldn't like .net/C# to be constrained as a Script ecosystem, yes it can do it, but it's a very small part of its capabilities. Unity employed C# at first for scripting, then looked to use it for more.

Going only to the Script Workflow path would be such a waste IMHO. Both would be ideal, but only if the cost of developing/maintaining the script part were low.

With the class extension scenario, you can "develop" and not only do small scripts and I think that's what people want and need. C#/.net is a good compromise (at least compared to C++) to develop big projects with good performances.

Working with an IDE like Rider and having a hot reload in the Godot editor upon compilation/save in Rider is quite suffisant I think. And if it's not good enough, we should work to improve this, because replicating the user experience of a full-featured IDE like Rider in the Godot editor would be a huge (if not poinless) effort.

theraot commented 8 months ago

In my opinion, the Extension class approach gives both a better workflow for the users and performance in runtime. Thus, that would be my preference. In fact, if C# classes get registered in ClassDB - unlike GDScript ones - I'd be inclined to do more C# code for Godot in the future (I've been doing GDScript only for over a year now).

I'm guessing having both would be too much trouble. In particular regarding whether or not one can switch a project from one to the other. If possible, perhaps some people would find it accesible to prototype with scripts, and then switch for extra performance. I wonder if that would be attractive for people not familiar GDScript (personally I would prototype with GDScript, and C# with extension class workflow seems like a good complement for it, also I remember the advice of migrating CPU intensive parts from GDScript to C# or C++ which makes sense for this and also supports the idea that we would expect performance from C#).

I'd rather not have C# using the script workflow only, in that case I would continue using GDScript, yet I understand that for some people GDScript is not an option (let us skip on why and if that makes sense), and I'm guessing this is more for them than for me.


About the listed cons, they do not seem too much trouble:

Can't easily replace in run-time, must delete a node and replace by a new one.

The editor will try to map property values when you change the type of a node from the context menu of the scene dock. Thus, I believe this is already mitigated.

Soft reloading will work, but may be troublesome in some unexpected corner cases due to this process being a bit more complex (need to add and remove from ClassDB).

It seems like the main benefit of using the script approach is some safety on this regard.

Yet, I'm confident crashes would be worked out in general.

Furthermore, if I'm not mistaken, there is already a solution for reloading, which might be improved or supplemented upon if necessary. And any work on this area might also benefit third party language projects, thus, even this is a challenge I think there is an starting point and it would be worth doing.


My main concerns would be on breaking compatibility, and how much of this work can actually be done in parallel. I was expecting this would be put off for a while.

IsaiahKelly commented 8 months ago

To me this is not script versus class extension, both are good, they are targeting different usage. ― @LoicBaumann

Yes and this is something many people here seem to be missing. It's not about making C# usability worse or better, the question is which is more important: performance or integration. Because everything has it's tradeoffs and you need to pick your priority.

Visne commented 8 months ago

Going only to the Script Workflow path would be such a waste IMHO.

This is exactly how I feel about it too. Having great performance, struct pointers, properly integrated add-ons, and the ability to patch internal parts of the engine (at least, the way it is stated in the proposal it seems like there's no other way to do that without extensions) all seem like amazing features that really integrate C# very nicely into Godot and that not having these options would be very disappointing. And while I think that having the integrated Godot editor have a debugger would be neat, I think it's not exactly a hot take so say that the overwhelming majority of developers using C# with Godot will be using an external IDE.

the question is which is more important: performance or integration

The way I see it extensions win on both, considering that extensions "[integrate] very well as if it was a native engine class" and the fact that add-ons will be normal C# classes instead of nodes with script files (this genuinely sounds horrible to me, honestly). You could probably argue that being able to overwrite parts of the engine is also better integration, but it would probably be better described as an extra feature.

definitelyokay commented 8 months ago

@reduz Thanks for clarifying.

I understand that Godot as a library is not the goal here, so no worries there.

The main advantage of this is that it costs much less to keep C# up to date for contributors.

I'm all for anything that helps contributors.

Godot is not a C# focused engine.

Would you care to elaborate? I'm sure a large chunk of former Unity users are watching to see what happens next.

There have been lots of posts and conversations talking about how Godot's C# support is not "first-class" (what that means exactly is hard to say, but the sentiment behind it is not hard to understand). As you can imagine, statements like this aren't very reassuring when that's been such a widely held concern within the community, and one I've been working hard to dispel.

I was under the impression that C# is intended to be just as officially supported as GDScript.

If the goal is to implement it the same as all other languages, that's fine β€”Β but I do think the C# community deserves an answer about how committed Godot as an organization is to C#.

moonheart08 commented 8 months ago

To me this is not script versus class extension, both are good, they are targeting different usage. ― @LoicBaumann

Yes and this is something many people here seem to be missing. It's not about making C# usability worse or better, the question is which is more important: performance or integration. Because everything has it's tradeoffs and you need to pick your priority.

I don't think this is the case with this specific "tradeoff", because one can always build an abstraction over the high performance API (if desired).

andy-noisyduck commented 8 months ago

@andy-noisyduck I understand the point you are trying to make, but a large majority of users and contributors want a performance focused C# integration. Its a matter of preference that dictated the priority, from there we work on making the usability as good as possible.

If that is the wider community consensus then so be it.

However, I worry where these stats are coming from. A claim on the majority of users seems broad given I know no one in the community that was actually asked their opinion (the only data I see is 3 hours worth of "quick poll" since this proposal was opened). The contributors are a always going to be a tiny subset of developers and I don't know how well they represent the wider developer community. The same is probably true of the percentage of developers that are active on Github.

Putting my thoughts on this specific proposal aside. I am concerned whether highly impactful decisions about C# are being made by a group of people that aren't representative enough of Godot's C# end users. How many of the core team represent active C# developers? How many of the core's team's views would align with commercial studios using C#? Looking in from the outside there doesn't seem to be anyone championing for either, and I would hate for decisions to be made based on an (unintentional) echo chamber.

SirLich commented 8 months ago

I am concerned whether highly impactful decisions about C# are being made by a group of people that aren't representative enough of Godot's C# end users.

The proposal is called "Godot C# Roadmap Discussion", and has been opened in the "godot-proposals" repo, the officially supported place to make proposals. If you click godotengine.org/proposals, you will find yourself here. Juan has also boosted the discussion on Twitter, and it will end up circulating elsewhere as well.

and I would hate for decisions to be made based on an (unintentional) echo chamber.

Please feel free to spread the word, and get more eyes on this topic! Unless it wasn't clear, the influx of unity users have sparked a large-scale discussion about C# the last few weeks. A discussion-based proposal seems like the perfect way to move forward!

andy-noisyduck commented 8 months ago

The proposal is called "Godot C# Roadmap Discussion", and has been opened in the "godot-proposals" repo, the officially supported place to make proposals.

Edit: Thanks for the downvotes team but I'm not sure they were warranted. Whilst I wrote the original comment late last night, and possibly not in the best frame of mind, the further comments today only seem to reinforce that this isn't a discussion at at all.

Reduz writes:

For one thing, the move to GDExtension is a must for maintainability purposes. That needs to happen regardless as it lifts a lot of long term pressure from current contributors.

A large majority of users and contributors want a performance focused C# integration.

This feels like the decision has already been made. What was the point in the poll? What was the point in the getting feedback? This seems like a discussion in name only.

paulloz commented 8 months ago

My actual two cents, now.

The emoji poll thing feels extremely awkward. It shifts a quite technical subject, that requires a somewhat intimate understanding of C#, Godot, and how the two are currently interoping, towards a simple yes/no question. A question on which, the source of truth for most people is what's written above and is, at least as I read it, heavily weighted on one option.

With that out of the way...

Let's first talk about how this looks. And it kinda looks like relegating the C# bindings to the same level as any other community binding (no offence to the awesome godot-rust community, or any other). A lot of users are worried about C# not being a "first-class citizen", and the answer is seemingly to move it from second to third class. Feels odd, but oh well, maybe I'm reading too much into this.

Now, about workflows. The script workflow is of course nowhere near as painful as described up there. I'm not sure the difference will be significative in the end, people adapt. I'd argue, if anything, having used other GDExtension bindings here and there, the class workflow does feel less integrated with the engine (which presses on my point above). There's one question about timing. Do we want all to throw off all the new users who jumped in for 4.0 (and, to a lesser extent, with the Unity situation)? Who's to say.

About performances. I don't think anyone anywhere is arguing we should aim for worse performances where possible. On the other hand, and unless I'm vastly mistaken, most people gathered in the last .NET contributor meeting agreed that it'd also be possible to address API issues while staying on the current implementation. So I find that presenting the move to GDExtension as the only definitive solution a bit disingenuous.

My next point is very pragmatic but, forgive me. What would be the actual plan? That's a fair amount of work to get this done, and there's not that many man-hours available on the .NET part of Godot. What happens while time is poured into this, does the implementation currently used in 4.x stay as is? Or do we continue to also maintain it, knowing full well whatever is produced has an extremely limited shelf life?

As an answer to some comments (bonus for you if you're still there), none of this affects work being done on supporting platform currently not supported (Android/iOS/Web/etc.).

Repiteo commented 8 months ago

@definitelyokay To be fair, that statement was in response to the idea of Godot being a library for C#, which would've absolutely been a C# focused decision. I wouldn't at all read into that as saying that C# isn't being considered for the engine; on the contrary, these proposals and the traffic being directed towards them by Juan himself shows me that this is very much something important to him and the community as a whole

CypherDoesStuff commented 8 months ago

Personally, as someone who is/was a Unity user, and also a fan of good performance. I will say honestly, I would just love having a good workflow with the editor.

I've never touched GDScript, simply because I'm most proficient in C#.

Performance wise, I feel like the raycast thing was pretty specific, Godots raycasts are pretty verbose compared to what I was used to. Though it seems like here using raycast nodes are preferred (correct me if I'm wrong though)

I've already ported over some procedural generation code for terrains, fast enough without using the fancy jobs system. If we can get the flexibility to use performance if we need it, great! But honestly, I just want to make games. Preferably in C#

The idea of C# being in-between GDScripts ease of use and C++ performance comes to mind.

Also this may be a stretch, but having hot-reloading would be pretty great. Nothing is worse than constantly seeing "reloading script assemblies"

theraot commented 8 months ago

On the matter of the poll, I do not worry about the reach since reduz post on the social network previously known as twitter usually get attention.

With that said, a drawback of the emoji poll is that there is no clear indication of when it is done. There would still be people arriving here and it might take a while to slow down (a day or two?).

Furthermore, a Google Forms poll or similar would allow to also ask and cross reference if the people answering are using C#, planning to use C#, would use C# if one of the presented workflows is implemented or the other, or would not use C# regardless; and also which workflow they prefer. Which you can't do on a single question, rendering the emoji poll insufficient.

Anyway, the presented paths require some understanding, and thus some explanation must prefix the poll regardless.

I'll reiterate that I was expecting this would put off for a while (again, because concerns of breaking compatibility and the work that contributors can actually do), thus I'd side on not rushing.

Repiteo commented 8 months ago

I'm wholly onboard with migrating to GDExtension. While what @paulloz says is true that this could be seen as going from second-to-third class, from where I sit C# would always be second-class if it continued to utilize the existing system. If GDExtension feels third-class, then the actual priority should be improving the UI/UX of the binding system first and foremost. If the team is going to present the binding system as a universal pipeline with which to connect and support practically any language, that process would ideally be as user-friendly and accessible as everything else Godot is known for

Visne commented 8 months ago

I feel like the poll was intended just to get a general feel of what the community thinks, whether it's heavily towards one side or maybe more split. I don't think it was intended as some kind of binding referendum.

Gamerfiend commented 8 months ago

I feel like moving C# towards a binding style that other languages use, while keeping it as first class supported would be the right move. An unspoken benefit to this, with the team working on support for this style of binding officially with C#, other bindings would benefit from any changes done to the API to better support it.

ChrisAbra commented 8 months ago

I think the extension method works better.

Personally i love the node based, compositional structure of Godot and one of the things i want to do most is extend it for specific usecases. e.g extending an Area3D into a health connected hitbox, or extending a Raycast3D to interact with such a hitbox. Ideally id do this in C#, where Interfaces make this code tidy, terse, easily managable, statically-typed and (to be honest) im just more comfortable with.

This means i want to write some of my core code in C# producing custom nodes i can compose my higher-level features and interactions out of. If i need some scenarios where they interact a little differently, that to me is where scripting comes in and GDScript is a genuinely great language for this.

I want to write core concepts and structures in C#, with custom nodes treated as if they were C++ modules, and if i (or a designer) need to make tweaks or customisations on those, GDScript makes sense for a Scripting language.

I would however echo paulloz' point above about a worry that pushing this out to a community-supported, extension project (Godot's OSS nature complicating this distinction notwithstanding), raises some worries for how dotnet support is funded. We wouldnt and couldnt expect the Godot Foundation to support any and all languages that wished to bind to GDE but at the moment it IS supporting "Mono" development and a lot of users would obviously like to see it continue doing so. Although i am sympathetic to the idea that making it significantly easier to support would aleviate this issue.

ryuukk commented 8 months ago

I don't like how Microsoft is slowly creeping in opensource projects, i knew it when mono encouraged you to have C# support, it was a trojan horse, and here we are today.. deeper integration

Make sure you are prepared to get things removed from you, remember folks

https://github.com/dotnet/sdk/issues/22247

let languages speak to you, do not start to speak to them, it'll make your future tied to them, with that comes with opinionated with self-interest requests

Oh, one last thing

https://code.visualstudio.com/docs/csharp/cs-dev-kit-faq

Don't forget to buy a licence if you work on commercial projects in VSCode ;)

For personal, academic, and open-source projects, the C# Dev Kit can be used at no cost. For commercial purposes, teams of up to 5 can also use the C# Dev Kit at no cost. For 6+ developers, those users will need a Visual Studio Professional (or higher) subscription.

https://code.visualstudio.com/docs/csharp/cs-dev-kit-faq

Remember Unity? why repeat the same mistakes?

ryanabx commented 8 months ago

My actual two cents, now.

The emoji poll thing feels extremely awkward. It shifts a quite technical subject, that requires a somewhat intimate understanding of C#, Godot, and how the two are currently interoping, towards a simple yes/no question. A question on which, the source of truth for most people is what's written above and is, at least as I read it, heavily weighted on one option.

Agreed. Even though I am for the change moving C# to GDE, an emoji poll is not definitive consensus. I don't think it will, in the end, be used in that way. I could be wrong though.

With that out of the way...

Let's first talk about how this looks. And it kinda looks like relegating the C# bindings to the same level as any other community binding (no offence to the awesome godot-rust community, or any other). A lot of users are worried about C# not being a "first-class citizen", and the answer is seemingly to move it from second to third class. Feels odd, but oh well, maybe I'm reading too much into this.

The optics are a little shaky, mostly because there's a lot of misunderstanding in how this affects C#. What I see the move to GDE as is an opportunity to focus manpower onto the core of the engine, instead of maintaining what could be considered as duplicate work to make C# work in the engine. The proposal seems to outline what would amount to generating automatic C# classes every time the engine has changes, which looks to me like as much first class support as you could have. The current situation involves having to remember to update the C# bindings every time a new feature is implemented, and that to me feels like third class support.

Now, about workflows. The script workflow is of course nowhere near as painful as described up there. I'm not sure the difference will be significative in the end, people adapt. I'd argue, if anything, having used other GDExtension bindings here and there, the class workflow does feel less integrated with the engine (which presses on my point above). There's one question about timing.

Yes, people adapt, and although this is anecdotal, I found the support to be lesser than the GDScript counterpart, and as a result stuck with GDScript. What would need to happen (and what I think will most certainly happen at the prospect of an exciting new project) is more intentional development work into GDExtension. I already think regardless of this outcome here that GDExtension should be more heavily prioritized in development.

Do we want all to throw off all the new users who jumped in for 4.0 (and, to a lesser extent, with the Unity situation)? Who's to say.

No, and as such, the new C# GDExtension MUST be labeled as EXPERIMENTAL, and the current mono module should be maintained until one of two things happens:

  1. The C# GDExtension reaches feature parity with the mono module, AND includes little to no breaking changes
  2. A new API breaking version of Godot releases (5.0)

About performances. I don't think anyone anywhere is arguing we should aim for worse performances where possible. On the other hand, and unless I'm vastly mistaken, most people gathered in the last .NET contributor meeting agreed that it'd also be possible to address API issues while staying on the current implementation. So I find that presenting the move to GDExtension as the only definitive solution a bit disingenuous.

I was part of that meeting, albeit a curious bystander (I didn't speak up). There are definitely other solutions to solving performance issues; however, they involve lots of development power that could be used improving a more universal aspect of the engine (GDExtension), while still achieving the same effect on C# performance.

My next point is very pragmatic but, forgive me. What would be the actual plan? That's a fair amount of work to get this done, and there's not that many man-hours available on the .NET part of Godot. What happens while time is poured into this, does the implementation currently used in 4.x stay as is? Or do we continue to also maintain it, knowing full well whatever is produced has an extremely limited shelf life?

The implementation in 4.x should stay as-is, with an experimental GDExtension C# plugin available, and my opinion is that the mono module should stay maintained for all of 4.x and be removed in 5.0. This requires a lot of man-hours unfortunately, and is very idealistic of me. The next best option would be to make the GDExtension as close to the mono module on the user-end as possible, and possibly ship an upgrade script with a new 4.x version.

As an answer to some comments (bonus for you if you're still there), none of this affects work being done on supporting platform currently not supported (Android/iOS/Web/etc.).

Always a good thing to note. I'm looking forward to C# platform support!

Delpire commented 8 months ago

Would C# users needs to do anything extra to get C# support in the future? Right now I have copies of different versions of Godot on my machine, and any time I want to start a project, I open the version of Godot I want, create a project, add a C# script, and my project gets generated and I am good to go. I can start writing and running code immediately.

If the new GDExtension requires going to some GitHub page, downloading something, and applying it per project, that would be quite annoying. Looking at the godot-rust documentation, it looks like a per setup project.

This might sound like nit picking, but for someone who likes to participate in many game jams and creates many projects for experiments, this level of friction would certainly be a downgrade from the current situation.

I'll also add that while I may not be the advanced case, I've never had a situation where C# performance was the issue. Any time I run into performance issues, its usually around physics, or draw calls.

Repiteo commented 8 months ago

@ryuukk Okay, let's not be wholly disengenous please

Don't forget to buy a licence if you work on commercial projects in VSCode ;)

This license is for the C# Dev Kit specifically, as the very page you linked points out:

The Dev Kit does not replace the existing C# extension but adds on top of the great language service features it provides.

And C# in Godot isn't some new thing being weaseled in opportunistically, it's been part of the engine since 2017 after a generous grant from Microsoft themselves; a grant given after the Godot team reached out to accomplish that very goal

You're free to feel however you like about Microsoft and/or Unity, but let's not act like C# is this boogeyman that'll suddenly make the engine worse by setting up an implementation that'll, by proxy, improve the binding system as a whole

ryanabx commented 8 months ago

You're free to feel however you like about Microsoft and/or Unity, but let's not act like C# is this boogeyman that'll suddenly make the engine worse by setting up an implementation that'll, by proxy, improve the binding system as a whole

Plus, this proposal actually removes most of the C# code from the main repo. If we were worried about the use of C# code in the repo, moving C# to a GDExtension is the perfect way to abstract it somewhere else.

And hitting on improving the binding system, this change helps developers of other bindings for godot (godot-rust, godot-lua, etc) by giving them a closer to first party support, because of the bigger investment into GDExtension. So if one didn't want to use C# but still wanted features of a mature programming language, it'd be easier to do so!

Delpire commented 8 months ago

If you favor script workflow (slightly better usability, but more limited in performance and extensibility), give this comment a πŸš€

After looking at the godot-rust documentation and watching a couple of Youtube videos, I think slightly better usability might be a massive understatement. I think this would be a significant hit to usability. Adding C# scripts to nodes is a very common task. Having the editor automatically create my .cs files for me is really nice. I can click on them in the editor to very quickly pop up Visual Studio (or VSCode) and edit them. I may not understand godot-rust very well, but it looks like a lot of friction adding rust scripts to nodes compared to the C# workflow.

I would be in favor of targeting the performance concerns of hot paths, until GDExtension has the kind of editor integration and usability that C# and GDScript already provides.

SilicDev commented 8 months ago

I'll preface this with the disclaimer that I've never used C# with Godot and have only worked a little with Godot's core and the GDExtension system.

First of all I'll echo what others have already said, however this ends up being implemented I doubt that it will be a net loss for the engine (unless stability takes a major hit) as not only could the workload on the C# part be reduced but also the GDExtension system gets some more eyes on it that might help it move away from being a "third-class citizen" of the engine and become more inline with core.

Personally I think both implementations (pure extension vs. scripting) have merit and so this comes down to preference of those that will use it. However it would be nice to see both eventually and if only to have a documented example on how to use these systems to create such extensions, particularly the Script(Language)Extension part could use some love.