Documentation generation for GDScript

[ThakeeNathees]

Describe the project you are working on: Working on documentation system for GDScript as part of GSoC 2020 The implementation : https://github.com/godotengine/godot/pull/39359 https://github.com/godotengine/godot/pull/41095

Describe the problem or limitation you are having in your project: The in-engine documentation is very useful as we could browse and search the whole API offline, without leaving the editor. But for those who design plugins and library for godot are lack of this feature, the users of it have no way to understand the API and how to use it.

previous discussion on this topic: [0] https://github.com/godotengine/godot/issues/4370 [1] https://github.com/godotengine/godot-proposals/issues/177 [2] https://github.com/godotengine/godot/pull/35716 [3] https://github.com/godotengine/godot/issues/23188 [4] https://github.com/godotengine/godot-proposals/issues/408

---

Describe the feature / enhancement and how it helps to overcome the problem or limitation: Implementing documentation support for GDScript with annotations(@vnen is working on it for the new GDScript parser https://github.com/godotengine/godot-proposals/issues/828).

• The parse tree has all the information about a property/function (name, data type, return type, setter, getter, default value, value of a const, ...) which will be extract to the DocData of a script and from the annotation It'll gain the the description, brief_description, tutorials, links, etc. and then the DocData will be updated with the Engine's helper window in real-time.

• With the documentation property of a GDScript, the code editor can show a brief description when a property/method is hovered and a description when auto-completing for the property/method. The description of an exported var could also be used as a tooltip[2].

• Since the Godot helper window supports bbcode, the documentations could contain images, colored text, alignments, bold, underlines, etc.

• An XML version of the documentation could also be generated for the plugin/library and it could be exported to HTML by the designer for his/her website.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

[WIP] doc comment (EDIT initial proposal change from annotation to doc comment)

[WIP] note: description and tutorial-link was hard coded since it isn't integrated with annotation yet.

[WIP] description when auto completing

any suggestions, modification, ideas, related features to include with this are welcome.

---

If this enhancement will not be used often, can it be worked around with a few lines of script?: No, It was a wanted feature for a long time and there is no simple work around

Is there a reason why this should be core and not an add-on in the asset library?: there is no way to implement this with an add-on

[dark-penguin]

# @short: Let the frobnication begin
#
# This function is tricky. To understand what it does, you have to read the [Tutorials] and
# experiment with it yourself.
#
# But here are a few examples:
# `frobnicate(5)` - does a foobar-type frobnication on an integer (which could also be a string)
# `frobnicate("5")` - does a completely different thing, for those coming from Javascript
#
# @tutorials - http://frobnicate.org/getting-started
# @category: Base types
# @tags: virtual
func frobnicate(something):
    # Implementation coming soon
    pass

# Implementation coming soon
func frobnicate(something):
    """
    Let the frobnication begin

    This function is tricky. To understand what it does, you have to read the [Tutorials] and
    experiment with it yourself.

    But here are a few examples:
    `frobnicate(5)` - does a foobar-type frobnication on an integer (which could also be a string)
    `frobnicate("5")` - does a completely different thing, for those coming from Javascript

    @tutorials: http://frobnicate.org/getting-started
    @category: Base types
    @tags: virtual
    """
    # Just pass for now
    pass

(EDIT: let's see how would it look with [ ] for links, ` ` for monospace and @ for tags Since it's for writing documentation, I think we can't do without a little markup, but this is just about all we need...)

[NathanLovato]

I still don't find the comment version hard to read or the other confusing (annotations are code blocks). I don't mind docstrings, I just don't see a need to change the language to support them just for that.

Again, about every language except python does docs in the code with comments, and I don't know that people have trouble with that.

In python I never use comments the way you do. The code is self explanatory most of the time, functions short... Comments are exceptional for my team. And they're typically a form of documentation. So we almost only have docstrings.

The example I put above is extracted from a larger project, our steering framework.

For an example of long files with GDScript doc comments see the Godot Nakama client: https://github.com/heroiclabs/nakama-godot

Thousands of lines of code with inline documentation per file. They're really clear and easy to read.

Note the proposal above is to differentiate doc comments from regular comments by having them start with ##

And the point of comments is that they are removed from the source code when building the game. Again, adding docstrings would be extra work and replacing something that is already here and that already works. I still don't see docstrings as an improvement in practice, even as someone who works with Python quite a lot. But that's just my view of course, others should participate too.

[dark-penguin]

I still don't find the comment version hard to read or the other confusing (annotations are code blocks). I don't mind docstrings, I just don't see a need to change the language to support them just for that.

I don't think we need to "change the language". Currently (in 3.2 git), putting multistring literals after function or variable definitions is already supported. I don't think we need to actually implement proper docstrings in the language - just use the same style for generating docs, which is what docstrings are created for, just in a different way. (By the way, they could be automatically removed when exporting the project.) I just think it's a really good syntax created for solving almost exactly this problem.

Again, about every language except python does docs in the code with comments, and I don't know that people have trouble with that.

That means, in other languages, there is no better solution than a "let's generate docs from comments" quick-and-dirty idea. In Python, there is, and we can borrow it with no drawbacks.

And the point of comments is that they are removed from the source code when building the game. Again, adding docstrings would be extra work and replacing something that is already here and that already works. I still don't see docstrings as an improvement in practice, even as someone who works with Python quite a lot. But that's just my view of course, others should participate too.

They are actually already there, so "why don't we just use them".

I actually understand your point, and it's really not such a bad idea - I just think we can do much better than that, and also possibly easier: @vnen and @ThakeeNathees wrote in the beginning that docstrings would be the easiest way, and comments are a pain to parse. I did not understand what's stopping them, and why did we move to discussing comments instead. So my point was "nothing is stopping us from doing that", and indeed others should participate. (The only thing I really did not like is @vnen 's idea to use annotations for this; that's a really bad idea from all perspectives.)

[dark-penguin]

In python I never use comments the way you do. The code is self explanatory most of the time

That got me thinking. Of course, I don't literally do pass # Just pass for now, but this reflects what I think about comments.

This is a game engine. There is a lot of prototyping going on, a lot of tinkering. I often use comments as short reminders for myself: "Just pass for now", "Don't rely on this", "TODO: this smells"... And I certainly don't want that to be "the official documentation for my project". So there should be a very clear, highlighted difference between "this is just a WIP reminder for myself" and "this is the documentation that goes in the final project". I illustrated that I would even make comments about docstrings, if I think I may want to rethink their wording later.

Even the wording is different: comments are for those who are looking at the code, and docstrings are out of context.

# Returns void and not "success" because we might want to make it a pure function later
func adjust_speed(delta: Vector3) -> void:
    """ Adjusts the player's speed for wind resistance """

The docstring explains how to use the function and what it does. The reason we want this issue is so that the user woudn't have to look at the source! The comments assume you're looking at the code next to them, so the wording is different, the purpose is different (at least sometimes), the intended audience is different - docstrings are for the end users of my addon, and comments are for the developers of my addon.

[vnen]

@vnen and @ThakeeNathees wrote in the beginning that docstrings would be the easiest way, and comments are a pain to parse.

We also said that annotations are easiest. And the counter-argument was that the parsing difficulty should be an afterthought, with a better syntax being priority. Also, @ThakeeNathees already found a way to parse comments, so it's not a big deal.

That means, in other languages, there is no better solution than a "let's generate docs from comments" quick-and-dirty idea. In Python, there is, and we can borrow it with no drawbacks.

I do believe is quite the opposite: Python doesn't have multiline comments, so they decided to strings instead. The way Python does it is also quite limiting in some ways (and a bit wasteful IMO, since those are available at runtime instead of stripped).

Honestly, putting docs after the declaration is a thing that only Python does AFAIK. The only "Python-like" in GDScript is the fact that it uses indentation for blocks, so we shouldn't be tied up on how Python does stuff. Since most languages uses comments before declaration for documentation, we gain more by following that (principle of least surprise).

This is a game engine. There is a lot of prototyping going on, a lot of tinkering. I often use comments as short reminders for myself: "Just pass for now", "Don't rely on this", "TODO: this smells"... And I certainly don't want that to be "the official documentation for my project".

That's why doc comments start with ##, so you can use a single # for regular comments.

[dark-penguin]

I also thought that annotations is "easy to do, hard to read", so it's a bad idea. Especially since the second-easiest thing is docstrings, which seems perfect.

Python keeps docstrings visible at runtime for a reason (though I don't think it's used very often), but we can strip docstrings during exporting.

That's why doc comments start with ##, so you can use a single # for regular comments.

That would still hurt visual distinction, especially when you have commented blocks which contain comments. And tags look like commented annotations. You can understand it if you look carefully, but not "immediately see" it. Could we maybe highlight them with a different color in this case? Ah, no, that would also wrongly highlight commented comments... I think mixing comments and embedded documentation is more "surprise" than defining a separate syntax for the documentation (and a very obviously separate one). And adding one more "#" is not "separate syntax" enough, it's more like "a single typo in the comments will lead to unpredicted behavior".

I'd still like to keep the "documentation for users, comments for developers" idea. And that documentation is "production code" with its own sources to be built from.

Could we maybe also have a way to create separate documentation files that get embedded into the documentation engine when they are discovered? Building docs from comments is good when you have little comments, but for extensive documentation (and with markup), you really want to separate it from the code.

By the way, I just thought, why not support both then?.. Same syntax internally (with @tags), but you could put descriptions either in comments (for those who have short informative comments) or in docstrings (for those who want more extensive documentation and keep the comment noise out, but not enough to warrant putting it in separate files)? Simply parsing "docstrings as well" is probably not so hard. In the style guide, you could say that one way is preferred, but the other way is available. Either "use docstrings, but you can also use comments for simplicity", or "use comments, but you can also use docstrings if you like them and your docs are too long".

[vnen]

You can understand it if you look carefully, but not "immediately see" it. Could we maybe highlight them with a different color in this case? Ah, no, that would also wrongly highlight commented comments...

They can be highlighted differently, at least the tags. It's not usual to comment a comment and when that happens it's usually not attached to anything else because you commented a whole block. Not sure if the highlighter can be that smart, but even without highlight it is not that unclear. Commented annotations also won't have double ## unless you commented a block already with commented code.

I think mixing comments and embedded documentation is more "surprise" than defining a separate syntax for the documentation (and a very obviously separate one).

Well, you're gonna be surprised by every main language out there other than Python... Some IDEs even show comments in tooltips by default, even if it's not really a language standard.

And adding one more "#" is not "separate syntax" enough, it's more like "a single typo in the comments will lead to unpredicted behavior".

It's not that bad because "unpredicted behavior" won't change how your code works. I bet many people won't even notice or will be pleasantly surprised by the result.

Also, you might be forgetting that multiline strings can also be used as comments. The only thing special is that they are multiline. Some people may also comment a whole block using triple quotes, instead of adding # to each line, and then have the same issue as you describe.

Could we maybe also have a way to create separate documentation files that get embedded into the documentation engine when they are discovered? Building docs from comments is good when you have little comments, but for extensive documentation (and with markup), you really want to separate it from the code.

We could potentially accept a doc format like an XML file, similar to what's done with the engine's class API. But keeping doc together with code is generally better because when you change code you can easily change docs. Also adding code don't require you to go to another file to describe it. So even if that is added as an option at some point, it should still have to work just with the in-script syntax.

By the way, I just thought, why not support both then?

Supporting multiple syntax styles from the get go sounds like sloppy design to me. In a way, multiline strings are also comments. OTOH we cannot then have a difference (even if minimal) from regular comments to doc descriptions (like we do by having ##). Still, that would require the strings to come before the declaration, not after, to at least have some consistency, otherwise you can use both syntax at once and it's not 100% certain what to prioritize.

[aaronfranke]

Since there has been a lot of mentioning of ##, I think I should point out that my original reason for proposing that syntax was to be similar to C#'s /// and to have a macro where typing ## auto-fills documentation fields for you to fill out. For example, if you have this code:

#
func foo_bar(baz) -> int:

Then typing the second # would auto-fill this:

## Foos the bar.
## @arg baz:
##  Baz.
## @return:
##  Returns an int.
func foo_bar(baz) -> int:

We can do some very basic inference based on the names and types, like C# IDEs do. Two-word function names can be split and have "s the " inserted in-between them, etc. Of course, users are expected to change these fields, they are placeholders.

I'm not against also supporting documentation that starts with #, and this would have the advantage of causing existing documentation written with # to show up. But maybe it's better to only support one syntax so that everyone's documentation looks the same.

Also, the above thought about # being good for backwards compatibility reminds me that using comments for documentation would also improve forwards compatibility, since you could take scripts from Godot 4.0 and use them in 3.2 and the documentation would do nothing but also not cause any errors. Also, you could also add documentation in Godot 3.2 projects and be ready for Godot 4.0.

[dark-penguin]

Showing popups based on comments is fine. Creating a documentation entry based on comments instead of proper documentation pages is a completely different story.

I just saw the built-in documentation for @GDScript, and yield in particular, and thought about how would we do the same. It would clutter the code, and over-clutter the comments. So we need some way to have documentation pages, which supersede derivation from in-code comments.

Still, that would require the strings to come before the declaration, not after, to at least have some consistency, otherwise you can use both syntax at once and it's not 100% certain what to prioritize.

That's the idea - I want "comments before, docs after"! That's flexibility! :) I don't think many people will actually do multiline comments using three quotes. As you said, not many people will use docstrings. It's easier to use a shortcut to comment the whole block. (And then make sure to not accidentally "uncomment" your ##-style documentation...). So the proposed idea for priorities is:

• Doc pages
• Docstring-style comments
• ##-style comments (and maybe even simple #-comments if there is no ## ?)

The reasoning is:

• If your docs are so large that you want to provide them separately, then they override anything else;
• If you want to embed your docs in your code, but still maintain them as "production material", distinct from comments, you would do it in docstrings;
• If you are just coding and not doing any of that, you get automatic comment popups. And you can even expand on that as you learn about them.

This way, we're accomodating for the "documentation separately" principle, but we're not hurting most users who don't even know what docstrings are, and just want automatic comment-based popups.

Basically, if you know what docstrings are and want to use them to support docs separately, you won't be hurt by in-comment docs; and if you don't know about doctrings, you are also not hurt, because you're not using them.

@aaronfranke backwards compatibility is fine either way - we already have docstring-style comments support in 3.2 (at least from git).

[NathanLovato]

@dark-penguin you seem to be making a lot of assumptions, about the amount of flexibility the target users of this feature might need, what beginners (who are beginners, even?) will understand or struggle with...

If you looked at the Nakama Godot API linked above, you'd see it's really not a problem at all to browse and edit the source code. And I think their code is exceptional for GDScript, where you tend to have short files due to the way the node tree works.

You should never design for flexibility your target users don't need. Good programming is the simplest solution that solves the specific problem at hand, not problems that people don't have (e.g. keeping their code reference separate from their code, which makes it harder to keep in sync).

We really need some plugin and framework developers to participate in here to talk more about their needs and concrete use cases.

[NathanLovato]

Allow me to precise my thoughts: a system for docs can be expanded later as enough people develop new needs. No need to increase the workload a lot upfront trying to offer flexibility. This is the best way to waste precious development time to create features people won't use. Flexibility and options also always comes at the cost of ease of use.

The need for docs generation is clear, much requested, to the point we've written an external tool in the meantime. There are users out there who write actual docs and we should ask them what their needs are, then find the simplest way to solve their problems. Not base proposals around principles, that lead to abstracting problems rather than solving them. Which is what is happening here, to me.

[dark-penguin]

Those were not purely assumptions; at least I want to use this system myself, and in my workflow, I would really want this particular "flexibility". And if the general consensus is that "no one uses docstrings", then it's safe to assume they won't be harmed by prioritizing docstrings over comments. Those are all suggested solutions to presented problems.

But yes, I am only one user, who admittedly only wants to do this. :) So please consider supporting after-definition docstrings as well, but what real users find most convenient first and foremost. :)

[YuriSizov]

I would prefer we didn't add two alternative syntaxes for the only benefit to be familiarity for one group of developers or another. Don't get me wrong, while I am biased in support of documentation comments, if we consider adding docstrings, I'd rather have either docstrings, or comments, but not both. These types of contesting ideas produce unnecessary ambiguity in coding styles, points of conflict that can never be resolved by the community itself. And it can be an issue for developers of all levels, because instead of adapting to one style you can develop your own and then suffer for it.

I am not agreeing with some of decisions made for GDScript style guide, for example, but that's on me, not on the official code style guide. But that means that the style that I've learned and am comfortable with may not be compatible with code layout and naming patterns of others. If we assume that I am building or joining a team, we'd have to work on these differences to some dissatisfaction of one of the parties. And then another team can have an entirely different view on the matter. But a language can enforce one vision and remove such points of conflict (or at least take that hit itself in forms of complains towards language maintainers). So I would prefer we design it to rely on one and only way of doing it.

PS. This argument does not apply to detached XML documentation, however. This approach has other benefits beyond personal preferences.

[Alvati]

I don't think "comments above, doc bellow" is the best when reading source. If you have comment -> signature -> documentation -> implementation

• reading for useage means reading the middle
• reading for refactor means reading middle -> top -> bottom (if you go purpose -> info -> actual code)

With documentation -> signature -> comment -> implementation

• reading for useage means reading the top (with a nice purpose -> syntax at that)
• reading for refactor means reading top -> middle -> bottom (if you go purpose -> info -> actual code)

---

I don't think documentation being compatible whith old code is a priority :

• Where the old comments comments or actual doc? Sure, it could format documentation nicely over night, or just as well make comments randomly appear on a documentation page.
• This will likely stay quite a long time. If Godot prospers for at least five more years, having documentation syntax be even slightly less good, is worse than lessened compatibility between two major versions.
• "Stumbling across the documentation system" seems like it would happen anyway. New users will likely look at a demo or at least a tutorial at some point, hinting at it. Old users will probably read the patchnotes for 4.0, and documentation support is not a small entry.

---

For "_..." functions and variables, I would suggest to display them slightly separated : ... -> Variables -> Dangerous Variables -> ... -> Functions -> Dangerous Functions With a message like "Avoid using/ calling these directly". This would encompass both pseudo-private functions and virtual ones (as they tend to be called by the engine directly)

---

Tweaks on some of the designs I like:

# Annotation style, but with \ instead of (" ")
# Similarly to how you multiline code
# Should be easy to implement (just add to the @tags text until you hit a
# newline whithout a \, tag start is just @)
# and takes litle extra characters to format properly

@brief Base class for different kinds of buttons.
@tutorial Check out this wonderfull tutorial : [url:about_blank] \
        , or this slightly more advanced one :\
        [url:https://github.com/godotengine/godot-proposals/issues/993]
extends Control

@brief Both the extend and class_name seem intuitive \
       for general class/ file tags, how do we handle this? \
       Silent override, error, never allow tags on class_name?
class_name BaseButton

@brief Emitted when the button was just toggled between \
       pressed and normal states (only if toggle_mode is active).\
       The new state is contained in the button_pressed argument.
@arg button_pressed if pressed  # defining signal args like that?
signal toggled

@brief Called when the button is toggled
@param button_pressed if we are pressed  #Obvious # Commenting the doc
@param button_pressed hello   # This line errors out, right?

func _toggled(button_pressed: bool) -> void:
    pass #Don't bother calling ._toggled() when you override

---

# C# style?, but with #! instead of ##
# Neat, but it doesn't allow to comment the doc as well
# not a big deal mind you
# I think I still prefer the one above

# Note: Here you inherit which tag you're talking about instead
# of having one complete symbol (\ eating up the newline and indent)

# Brief as the default tag, no need to mention it
#! Base class for different kinds of buttons.

# If no @ is present, just append to the thing we are writing about
#! @tutorial Check out this wonderfull tutorial : [url:about_blank]
#!           , or this slightly more advanced one :
#!           [url:https://github.com/godotengine/godot-proposals/issues/993]
extends Control

# Here is a good example, the brief is the default
# and subsequent lines just get appended (with indent stripped to 
# a single space, if you need multiple spaces, put them outside
# of a line break)

#! Both the extend and class_name seem intuitive
#!       for general class/ file tags, how do we handle this?
#!       Silent override, error, never allow tags on class_name?
class_name BaseButton

#! Emitted when the button was just toggled between
#!       pressed and normal states (only if toggle_mode is active).
#!       The new state is contained in the button_pressed argument.
# defining signal args like that?
#! @arg button_pressed if pressed
signal toggled

#! Called when the button is toggled
#Obvious # Commenting the docs slightly less well
#! @param button_pressed if we are pressed

# This line doesn't error out anymore, it gets appended to the:
# "if we are pressed" to make "if we are pressed hello"
# Or just error out if you want, it may cause bad practices
#! @param button_pressed hello

#! @brief Do you allow for late briefs like that?
#!         if so, and you do decide to error out when shifting
#!         to a tag you've been in, you have to either add an exception,
#!         or say that it's ok as long as the text for the tag is empty

func _toggled(button_pressed: bool) -> void:
    pass #Don't bother calling ._toggled() when you override

Note : Try to imagine those with syntax highlighting, the existing one plus some color for the @tags (maybe different shades per tag? to make them faster to look for?)

Also, the second version has more text, to explain how it would work and ask questions inherent to this approach.

[YuriSizov]

Try to imagine those with syntax highlighting, the existing one plus some color for the @tags (maybe different shades per tag?

There is your problem with your first suggestion, though. We are getting annotations in 4.0's version of GDScript, and those tags syntactically mix with them. Without syntax highlighting it would be harder to read, and for syntax highlighting to work properly, parser would need to understand if it's an annotation or a documentation "tag".

I also don't see a point in adding comments to the documentation or in commenting out parts of it.

[NathanLovato]

I also don't see a point in adding comments to the documentation or in commenting out parts of it.

Agreed.

The proposal above makes writing docs more complicated than it needs to be. If you write actual API docs, you'll see you don't need that much functionality or special highlighting. You spend way less time to write and update code docs than to design code in the first place, so you don't need much functionality there.

I'd invite everyone who's commenting on this proposal to write some real-world API docs in at least two different languages. You'll quickly realize that a simple solution works well, that it doesn't make the code hard to read or to manage.

Python and Emacs Lisp, where the convention is to document every function or module, sometimes with extensive inline code documentation, just have docstrings, pretty much no special markup, and it works well. JSDocs works great too, and it's just regular comments with some @keyword -. Same for C#, although it uses XML and text editors typically give you a template with the tags. But IIRC besides that, it's regular comments.

Again, there should be a need to justify more complex syntax or alternatives, like simpler solutions causing concrete problems in production.

[nonunknown]

guys I was testing here and variables starting with _ are still shown in the Property Description section, and this shouldnt happen

I think a special word should be make, because there's some situation where the user wants the "private" variables to be shown so something like this at the top of the script:

extends Node
class_name Test

## ignore _

[Calinou]

I think a special word should be make, because there's some situation where the user wants the "private" variables to be shown so something like this at the top of the script:

The documentation generation system is intended to document public APIs, so private properties and methods shouldn't be displayed there. I would just exclude properties whose name start with an underscore. However, we can't do that for methods as virtual methods start with an underscore.

[YuriSizov]

Well, GDScript doesn't have virtual methods, so we indeed can do that for methods. We could also add something like an annotation to denote GDScript methods as virtual, if that's an important designation. Then we could include them in the docs regardless.

[NathanLovato]

Documenting virtual functions is pretty important yes. For instance, we worked on some AI framework and you need virtual functions for users to plug into the system. We write them with a leading underscore for consistency with Godot. We do have something similar where we mark functions as virtual with a comment above the definition, to distinguish them from pseudo-private stuff.

[vnen]

@nonunknown please open an issue on godot/godotengine about this. Comments in a closed proposal will likely be missed and forgotten.

[Calinou]

For virtual annotations, there's already a proposal: https://github.com/godotengine/godot-proposals/issues/1631

[nonunknown]

@vnen there's a proposal for that already create by @ThakeeNathees which is #4370 So that makes no sense! unles I report as bug instead!

[vnen]

@nonunknown but it is a bug: variables starting with _ are considered private and should not show in the help window. This is what the docs say: https://docs.godotengine.org/en/latest/tutorials/scripting/gdscript/gdscript_documentation_comments.html#documenting-script-members

If it's not working like that then it's a bug.

[nonunknown]

ok awesome then, I'll report! thanks.

[marcinn]

Why not markdown? It is highly readable as a pure text. I started documenting with bbcode, but comments are unreadable in source files.

[YuriSizov]

@marcinn Because we already have a format for documentation markup, we're just reusing it, and markdown also doesn't work well with GDScript comments syntax.

[marcinn]

Thanks for the explanation, @YuriSizov.

Adding the markdown parser library shouldn't be difficult. The syntax is a problem. So... how about introducing python-style docstrings? They will not clash with the markdown syntax, and docstrings will gain readability. Both styles (bbcode and markdown) can live together until G5 or G6 (until bbcode is removed).

[YuriSizov]

BBCode is not going to be removed. And docstrings are not going to be added. You can still use multiline strings to document your code if you want, but this syntax that is implemented here is for the built-in help, so it must be written in a way compatible with that. There isn't much benefit in adding multiple ways to describe classes with multiple parses to maintain.

[marcinn]

There is a huge benefit from markdown - readability. As a human you aren't forced do parse and render bbcode tags in your head when reading and maintaining sources. EDIT: Also note that bbcode is a markup used in fairly old forum engines, while devs use markdown mostly.

[YuriSizov]

There is a huge benefit from markdown - readability. As a human you aren't forced do parse and render bbcode tags in your head when reading and maintaining sources.

Yes, and this is not for human readability. It's for the built-in documentation generation. A different medium, that has specific inner workings. We need special markup that is designed to work for the purposes of the built-in documentation. Even if changing all of our markup language to (a more limited) markdown would be up to consideration, that's not something we can change now, for Godot 4, without introducing excess and confusion with a secondary system as you propose.

That said, most of the documentation can be written with simple ## Description syntax. You only lose readability if you start to add more elaborate formatting to it.

Also note that bbcode is a markup used in fairly old forum engines, while devs use markdown mostly.

Prejudice aside, markdown is a limited language, and its only expansion is to embed HTML. Which makes it even harder to parse and more prone to vulnerabilities. Godot uses bbcodes for rich text markup throughout its rich text controls, and you can create your own tags for custom effects and extended functionality. It serves different purposes. Documentation is built on top of that and leverages its powers.

[marcinn]

Yes, and this is not for human readability. It's for the built-in documentation generation

I'm trying to say that Markdown (properly extended) will solve both things.

that's not something we can change now, for Godot 4,

That's why I proposed a period of N years, where two markups will live together, then depreciate the old one. I believe that most devs would pick Markdown.

You only lose readability if you start to add more elaborate formatting to it.

That's my problem. I focused on documenting my addon, and I lost a lot of readability. The square brackets syntax used for classes, enums, etc, is pretty fine. But the problem is with tags like [code], [b], newlines with [br], missing paragraphs, and lists, which never worked for me (note that plain text asterisks and dashes are rendered in a single line). I have mess in the doc, or mess in the source file. And I can't have both good enough until something like Markdown will be available (because of bbcode syntax).

Godot uses bbcodes for rich text markup throughout its rich text controls

Ah yes, I read about this yesterday. But it's a different purpose than writing docs, don't you think?

Documentation is built on top of that and leverages its powers.

Markdown syntax can be extended in a similar way, I think.

---

Yuri, my goal is not to force you to do things you don't want. I'm just asking and suggesting solutions from a dev perspective (and as far I understand, the engine and editor are created for devs). There are issues with bbcode in docstrings. Some can be solved by fixing bugs (like extra whitespaces in docs which comes from [br]), but others will never be solved because of the bbcode syntax. I can create proposal and some prototype, if there is a chance to improve Godot on that field.

Thanks for your time.

[YuriSizov]

I'm trying to say that Markdown (properly extended) will solve both things.

It will not, for the reasons I provided.

That's why I proposed a period of N years, where two markups will live together, then depreciate the old one.

We do not practice having two competing systems that do the same thing. This is not user-friendly, and it creates unnecessary maintenance burden. And that also assumes that we want to remove "the old one". You propose a long-term commitment to a hacky solution, while creating an extra point of friction for users who would have to choose between different systems. That's not to mention that we would have to either parse MD ourselves, which is not a trivial task, or integrate yet another library for a very minimal feature.

Ah yes, I read about this yesterday. But it's a different purpose than writing docs, don't you think?

It's relevant, because this is the basis for the entire system at hand. We can natively and efficiently support documentation comments with the same capabilities that are available in Godot's own documentation and in Godot's own rich text controls. And it gives us tools for customization that Markdown doesn't have.

Markdown syntax can be extended in a similar way, I think.

It cannot, it doesn't have a capacity for extension in the same way.

There are issues with bbcode in docstrings. Some can be solved by fixing bugs (like extra whitespaces in docs which comes from [br]), but others will never be solved because of the bbcode syntax.

These are not issues. These are personal preferences. Verbosity of the syntax is hardly a blocker for most devs — some other systems are even more verbose (C# for example, which uses XML for their documentation generation). Just because you prefer Markdown and are used to Markdown, doesn't mean it's the best option that we have here. And I'm providing you rationale for the current system (if the preceding discussion is not enough), and explain technical limitations of Markdown that we would have to overcome or live with.

You can open a proposal, but there is a number of reasons why any change is unlikely, and your stop-gap solution would be even worse. Just to reiterate:

• The existing system comes at a relatively low cost for the engine, leveraging the power of the existing engine systems for rich-text content. This allows us generate rich documentation pages on par with built-in documentation, without compromises or limitations for users.
• This feature has been discussed for a long time, and various syntax options have been considered. It's naïve to come to the discussion at this point with "Why not markdown" as if no other solution has been considered, but I'm fine discussing this anyway so you can have a form of closure instead of the question hanging forever answered.
• The implementation is still very much readable, even though it can be unfamiliar to some. I've used quite a few programming languages, and they all have different syntaxes for documentation comments. Most don't have any formatting capabilities, actually. So IMO for most developers it will be sufficient to just write plain text descriptions, with very occasional formatting.
• Having two systems at the same time is a bad solution to any problem. It creates more frictions to users, who end up arguing which system is the best instead of writing code and making games. You can see examples of this in practically any programming language, starting with banal tabs vs spaces. Godot 4 only just released, so with your suggestion the entire lifecycle of this major release would have two competing implementations, distracting users from their work with rather pointless conflicts.
• Having two systems also multiplies several times the amount of maintenance. That's even ignoring the complexity of full Markdown support, whether first part or third party. We have to practice restrain when it comes to adding maintenance costs, and when it comes to adding third-party dependencies for that matter. This applies to absolutely any proposal, and is beyond this conversation.
• All that on a premise that Markdown is superior, while it clearly lacks customization options that we need with our "domain-specific" markup language. And while it conflicts with the comment syntax in GDScript (which only has single-line comments, by the way). I would say, that RST from Python's docstrings would have a higher chance of adoption, as we already use this for online documentation. But then again, it requires a parser.

In short, it's too late. And you don't propose a clearly superior solution. And your stop-gap would be all sorts of problematic. And it's all based on a personal dislike or unfamiliarity for bbcodes.

[marcinn]

It is not about personal preferences. It is about user experience and using right tools to do the job. Bbcode is not the right tool for docstrings created by devs. That's all, and it is quite simple. This is a markup ceeated for bboards/forums, as an safe alternative to html. As a result docs will not use bbcode and will not be good enough (readability of generated doc), or will be unreadable in sources.

Your answers are based on wrong assumptions. Markdown will solve issues mentioned by me, there is always good time to discuss about improvements, and nobody here wants to force these changes in 4.x.

And it is not about liking/disliking/knowing the syntax of bbcode, but about readability of the sources (and generated docs at the dame time). I wrote about this few times.

Regards, Marcin

PS. To be clear - bbcode in richedittext is the right tool for the job. Markdown can introduce problems here. Docstrings are different, they aren't part of the runtime, so the engine may have no support of MD. Just the editor.

[vnen]

There is no point discussing when a PR for this was merged over 2 years ago. The time for suggesting changes for this implementation is already passed.

Anyone can open a discussion (or an actual new proposal) to bring forth their suggestions and arguments.