Documentation generation for GDScript

[ThakeeNathees]

Describe the project you are working on: Working on documentation system for GDScript as part of GSoC 2020 The implementation : https://github.com/godotengine/godot/pull/39359 https://github.com/godotengine/godot/pull/41095

Describe the problem or limitation you are having in your project: The in-engine documentation is very useful as we could browse and search the whole API offline, without leaving the editor. But for those who design plugins and library for godot are lack of this feature, the users of it have no way to understand the API and how to use it.

previous discussion on this topic: [0] https://github.com/godotengine/godot/issues/4370 [1] https://github.com/godotengine/godot-proposals/issues/177 [2] https://github.com/godotengine/godot/pull/35716 [3] https://github.com/godotengine/godot/issues/23188 [4] https://github.com/godotengine/godot-proposals/issues/408

---

Describe the feature / enhancement and how it helps to overcome the problem or limitation: Implementing documentation support for GDScript with annotations(@vnen is working on it for the new GDScript parser https://github.com/godotengine/godot-proposals/issues/828).

• The parse tree has all the information about a property/function (name, data type, return type, setter, getter, default value, value of a const, ...) which will be extract to the DocData of a script and from the annotation It'll gain the the description, brief_description, tutorials, links, etc. and then the DocData will be updated with the Engine's helper window in real-time.

• With the documentation property of a GDScript, the code editor can show a brief description when a property/method is hovered and a description when auto-completing for the property/method. The description of an exported var could also be used as a tooltip[2].

• Since the Godot helper window supports bbcode, the documentations could contain images, colored text, alignments, bold, underlines, etc.

• An XML version of the documentation could also be generated for the plugin/library and it could be exported to HTML by the designer for his/her website.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

[WIP] doc comment (EDIT initial proposal change from annotation to doc comment)

[WIP] note: description and tutorial-link was hard coded since it isn't integrated with annotation yet.

[WIP] description when auto completing

any suggestions, modification, ideas, related features to include with this are welcome.

---

If this enhancement will not be used often, can it be worked around with a few lines of script?: No, It was a wanted feature for a long time and there is no simple work around

Is there a reason why this should be core and not an add-on in the asset library?: there is no way to implement this with an add-on

[YuriSizov]

I'm not sure I like the proposed syntax. It's pretty verbose, not easy to read and it has a downside of replacing comments completely for code documentation

I'd argue, that an inline documentation should be completely readable without any external tools. Currently, text is all over the place, aligned randomly and spread into several annotation statements. It'd be better to use specially formatted comments instead of such annotations, like shown here, for example. This way the descriptions are perfectly readable without any syntax highlight.

[willnationsdev]

@pycbouh

Well, first of all, he's saying that he can get the necessary information directly from the parse tree. And for that matter, you could also get it from the generic Script reflection methods (like get_method_list() which will also tell you the same information. If you're going to have a doc-comment like the one in the example, then you can just use static typing since the doc information would become implied from the given type hints.

I don't really see a significant difference between...

func foo(bar):
    """
    @method foo
    @param bar extends Node
    @return extends Reference
    """
    return bar

and what could already be there in vnen's current version:

func foo(bar: Node) -> Reference:
    return bar

Cluttering the code with annotations that don't actually contribute any meaning would be wasteful. In fact, this is better, since it catches the error that is ignored in your sample, i.e. can't return a Node if expecting a Reference. It's more concise too.

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

@param bar:
    some description
@return:
    some description
func foo(bar):
    return bar

Here it is for the class definition:

@brief_description:
    An image processing library for Godot
@description:
    This library contains multiple image processing algorithms which
    works with [Texture] node. Call [method ImageLib.to_gray] to convert
    image texture to gray scale.
    Here is a demo screenshot [img]res://addons/image_lib/demo.jpg[/img]
    [codeblock]
    func _ready():
        var im = ImageLib.ImageRGB.new()
        var gray = im.to_gray()
    [/codeblock]
@tutorial:
    http://gd-image-lib/tutorial.html
class_name ImageLib

WDYT?

Also, personally, I would much prefer it if the information by which we derive the DocData information comes from the core Script interface, that way we can have a single framework by which to generate class documentation that is consistent between all programming languages. That is one of the goals of #22 anyway which has gotten a lot of support from the community.

[aaronfranke]

Where does "Warning: don't use ImageLib.some_old_method" come from? This text does not exist in the example code, but it does exist in the example output. Also, your example doesn't follow the GDScript style guide since it uses ' for strings instead of ".

Aside from that, I think it would be nice to have something similar to the discussion here.

# Called once per frame.
# @param delta - Time passed since last frame.
func _process(delta):

This way, existing comments above methods would automatically be usable as documentation, and it would be very simple to write. The first part of the text would be interpreted as the description (or maybe it should be the brief description), until an @something changes the context.

Additionally, I would love if there was a macro for easy generation of documentation blocks, similar to /// in C#. Perhaps typing ## before a function should create something like this, and then users would be expected to fill out the lines with the description:

##
## @param delta
func _process(delta):

[willnationsdev]

Is vnen planning on having a distinct separation between language annotations and comment annotations? I mean, if that's something the community is willing to do, then I'd be good with @aaronfranke's proposal too, since it allows people to differentiate between comments (#) and doc comments (##). But I'd prefer that as much information be inferred from the type hints in the actual code so people don't have to re-iterate information between their comments and their code.

[YuriSizov]

@willnationsdev

If you're going to have a doc-comment like the one in the example, then you can just use static typing since the doc information would become implied from the given type hints.

I've only linked the example to share the look I'd go for, not the features that it describes. When I think of inline documentation in GDScript I think about descriptions, and not class and type hints. We don't need those, we are not JavaScript, we have a type system and a proper way to declare things.

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

Yes, that would work as well. Though vnen has not yet confirmed that such syntax will be available (as far as I remember the annotations issue). And the downside here is that this will still completely replace the existing comments, instead of relying on them. It would be nice to have existing, simple comments be interpreted as class, property and method descriptions out of the box.

[Calinou]

GDScript Docs Maker is worth a look at too.

[vnen]

The thing is that comments are a hassle to parse. The tokenizer currently can simply discard all the comments. It can be done but I think it might be an overcomplication of the system.

The only advantage of comments is that doing:

# Player speed in pixels per second.
var speed = 100

Is a bit easier in the eyes than:

@description("Player speed in pixels per second.")
var speed = 100

But then there are trickier things: brief/long description for class, tutorial links, probably something else that I'm missing.

When it comes to functions, this could potentially be solved with docsrings:

func clamp_speed(speed: float) -> float:
    """Clamps the speed value into the game limits and returns the result

    Parameters:
    speed: The speed value to be clamped.
    """

Which also needs a special parser for this, but it's more convenient because the parser is seeing the string there, so it can be extracted while parsing the function. (I believe that the docs currently don't support description for parameters, so the whole string could be used as description). But then this would only work for methods (maybe classes) and not anything else.

Using annotations is the "easy way out" because they are already parsed and their contents can be easily retrieved.

---

If you really did want to have custom information associated with each detail, then I'd recommend vnen supporting a syntax that allows you to switch to a block that directly supplies the information as bare words, and the syntax highlighter can just auto-interpret it as string, optionally in an indented sub-block.

Yes, that would work as well. Though vnen has not yet confirmed that such syntax will be available (as far as I remember the annotations issue).

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

---

I still think annotations is the better compromise because they're much easier to parse. We can probably improve the look somewhat, maybe even with the highlighter, but it's simpler to use annotations than something else.

@ThakeeNathees if you have some suggestions to improve this based on feedback (and you are comfortable implementing them), feel free to update the proposal. I'm personally okay with it so far.

[ThakeeNathees]

• comments as documentations are only for some thirdparty tools like Doxygen. bringing it to the language parser will simply break everything. for an example (without ignoring the comments)

  func _ready():
  if true:
      print("true", ## not the second arg
          ## still not the second arg
          "i'm the second arg")
  ## now the if doesn't have an else
  ## and there is an else below without an if
  else: ## this comment breaks the indentation of the next print
      print("false")

  I'd argue, that an inline documentation should be completely readable without any external tools.

yeah, we can't go with the external tools, so can't use comments.

• Using string literal is easier to implement and would work for function and maybe for classes (even no way to distinct between description, brief_description, tutorials) but what to do with member variables, signals, enums, constants ?? using docstring for functions and something else for enums brings inconsistency, (if you have any idea on how to resolve this, a mock up pice of code would be helpfull).

• for those reasons annotation is the better deal, and the documentation can become the part of the reflection. like @vnen said we could improve the look of the annotations, may be use editor code folding, sinppet generation, etc.

[YuriSizov]

@vnen

I still think annotations is the better compromise because they're much easier to parse.

I understand what you are standing for here, but your argument is completely about how easy or hard each proposal is to deal with internally. While a small, beautiful, easy to maintain implementation is what any programmer would want, we shouldn't make every decision about language based exclusively on how easy it is to implement.

Regarding complicating the parser, though, usually documentation comments use some special tokens, that can be parsed as both comments and something else. While it would complicate some things, it shouldn't make parser unbearable to deal with.

@ThakeeNathees

yeah, we can't go with the external tools, so can't use comments.

You completely missed my point. Any code editor with its syntax highlighting and IDE-lite features is what is here referred to as "external tools". This includes the built-in Script editor and anything attached to the GDScript language server. My point is that you shouldn't need to use any editor to read those docs when dealing with code, and without a powerful tool by your side the syntax proposed by you is hard to parse by a human.

Also your example is not how comments are usually used for documentation. Look at JSDoc

/**
 * Represents a book.
 *
 * @constructor
 * @param {string} title - The title of the book.
 * @param {string} author - The author of the book.
 */
function Book(title, author) {
}

and documentation comments in C#

/// <summary>
///    This method changes the point's location by
///    the given x- and y-offsets.
/// <example>
///    For example:
///    <code>
///       Point p = new Point(3,5);
///       p.Translate(-1,3);
///    </code>
///    results in <c>p</c>'s having the value (2,8).
/// </example>
/// </summary>

public void Translate(int xor, int yor) {
    X += xor;
    Y += yor;
} 

In my opinion you can read those easily without any tools, given that they are properly formatted (I've disabled syntax highlighting on purpose). And what is proposed here is too busy to be used outside of Godot Editor, if even there.

---

To both of you, guys, let's make sure that the implementation is easy to parse for a human first and foremost, and for a machine only second. We shouldn't sacrifice our ability to read code for a prospect of documentation 🙂

[YuriSizov]

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

What if we allow annotations to apply to comments, though?

@description:
  # Player speed in pixels per second.
var speed = 100

So it still is applied to the block and not uses whatever comes next as its string content.

Would that be easier to parse? Because to me it would be easier to read, probably.

[willnationsdev]

Note that the syntax proposed there would conflict with this proposed here. There it applies the annotation to all statements in the block while here it uses the block as a string argument for the annotation. Those cannot coexist.

If the normal behavior is something like this...

@export:
    var a := ""
    var b := 0

...resulting in a and b both being exported, and for that reason, we cannot support this...

@description:
    Some information.
    Some more information.

...because it would be equivalent to...

@description("Some information.")
@description("Some more information.")

...then having commented-out results wouldn't make a difference since multiple lines of comments would still generate multiple annotations. For example...

@description("# Player speed in pixels per second.")
@description("# Example: [code]velocity = delta * speed[/code]")

However, couldn't you resolve the issue by just saying that, for these types of documentation-related annotations, you instead build the results of multiple same-named annotations by joining them with newlines? In which case, the color block style would be consistent again. However, you'd still need to have the block use the comment syntax that @pycbouh proposed in order to keep it from registering the text as full-on property declarations and whatnot. Either that or allow the block to just be a big docstring.

@description:"""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
"""
var speed = 100

@description:
    # Player speed in pixels per second.
    # Example: [code]velocity = delta * speed[/code]
var speed = 100

The below example looks a lot cleaner.

Edit: removed reference to needing backslashes in docstrings since...you don't need them (forgot for a sec). In which case, either strategy really works, honestly.

[vnen]

However, couldn't you resolve the issue by just saying that, for these types of documentation-related annotations, you instead build the results of multiple same-named annotations by joining them with newlines? In which case, the color block style would be consistent again. However, you'd still need to have the block use the comment syntax that @pycbouh proposed in order to keep it from registering the text as full-on property declarations and whatnot. Either that or allow the block to just be a big docstring.

Well, adding a feature (block annotations) with many exceptions baked in right from the start sounds like the wrong way to go IMO. Either we do one or another (though I'm not convinced by either yet).

Also, I don't see a lot of difference between this:

@description:"""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
"""
var speed = 100

@description:
    # Player speed in pixels per second.
    # Example: [code]velocity = delta * speed[/code]
var speed = 100

And this:

@description("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]
""")
var speed = 100

You can also close the string in the last line:

@description("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]""")
var speed = 100

We can also change the annotation name to be shorter and less noticiable:

@doc("""
    Player speed in pixels per second.
    Example: [code]velocity = delta * speed[/code]""")
var speed = 100

Which is not very far from a comment and it's much easier to deal with code-wise.

For example C# uses the XML tags, which is some required structure to deal with docs. In GDScript we can use annotations for the same effect.

[YuriSizov]

Another point to consider: languages using pure comments for inline documentation allow for things like attributes and code-affecting annotations to be visually different.

Compare this:

/// <summary>
///    This method changes the point's location by
///    the given x- and y-offsets.
/// <example>
///    For example:
///    <code>
///       Point p = new Point(3,5);
///       p.Translate(-1,3);
///    </code>
///    results in <c>p</c>'s having the value (2,8).
/// </example>
/// </summary>
[System.Runtime.InteropServices.DllImport("user32.dll")]
[CustomAttribute("argument", Flag=true, AnotherFlag=false)]
public void Translate(int xor, int yor) {
    X += xor;
    Y += yor;
} 

To this:

@description("""
    Set player's speed in pixels per second.
    Example: [code]set_speed(velocity * delta)[/code]""")
@tutorial("https://example.com/set_speed_tutorial.html")
@connect(input_control, "value_changed")
@custom_annotation("argument", true, false)
func set_speed(value : float) -> void:
    speed = value

It may not be the worst case, but it's an important idea to consider.

[Zylann]

I dislike the use of annotations for this. See my comment about it: https://github.com/godotengine/godot/issues/20318#issuecomment-608302968

Also the fact this is only about GDScript, while Godot supports C#, NativeScript and other, it means something more should still be provided for Godot to have script docs like this.

I started an doc extraction addon recently but Calinou made me realize a hidden feature (access to AST through language server?) which allowed to implement this https://github.com/GDQuest/gdscript-docs-maker

Unrelated note: the examples imply the use of class_name to document a script, but class_name is optional. That means to document a class you would have to annotate... in the void? Unless script-wide annotation already works without, but I'm doubtful about it cuz you could start a script with nothing but a sub class definition or a member, which makes such annotation ambiguous. Perhaps that should also be mentionned on the annotation GIP, but such a problem would arise only for doc so far.

[willnationsdev]

Also the fact this is only about GDScript, while Godot supports C#, NativeScript and other, it means something more should still be provided for Godot to have script docs like this.

This is really important imo. It won't do much good if the only language that can take advantage of documentation generation is GDScript.

I dislike the use of annotations for [documentation].

In this sense, I think it comes down to whether enough people absolutely don't want annotation-based in-language documentation such that it's worth the effort to implement an alternative, more complex parsing workflow for comments.

So, we pick from one of the options below, unless someone else has another idea, yeah? Should probably send out a poll or something (which highlights how the decision is one between implementation simplicity/maintainability and usability/readability).

@doc("""
Player speed in pixels per second.
Example: [code]velocity = delta * speed[/code]""")
var speed = 100

# Player speed in pixels per second.
# Example: [code]velocity = delta * speed[/code]
var speed = 100

[ThakeeNathees]

In this sense, I think it comes down to whether enough people absolutely don't want annotation-based in-language documentation such that it's worth the effort to implement an alternative, more complex parsing workflow for comments

yeah, I've discuses with @vnen and planed to implement it with comments by collecting comments separately and when compiling it, apply the comments as documentation string to the properties.

so the documentation will look like

## Player speed in pixels per second.
## Example: [code]velocity = delta * speed[/code]
var speed :int= 100

## Set player's speed in pixels per second.
## Example: [code]set_speed(velocity * delta)[/code]
func set_speed(val:int)->void:
    speed = val

## this signal emitted when ... 
signal time_out

not sure how class description , brief description, and class tutorials would be. any mock piece of code with those would be helpful.

@willnationsdev @Zylann @pycbouh @aaronfranke

[willnationsdev]

@ThakeeNathees If it were following conventions from other comment-based dynamic programming languages, (PHP, Python, etc.), then I'm thinking you'd have @-based annotations within comments that separate fields, and then if none are given, it just all defaults to a summary/description value.

# Full version
## @brief A summary here
## @description Blah blah something descriptive here and
## continues between newlines until another doc annotation.
## @tutorial https://website.com/tut
var speed: int = 100

# Abbreviated version with only description
## Blah blah something descriptive here and
## continues between newlines until another doc annotation.
var speed: int = 100

# Combo version with description and other custom doc comments.
## Blah blah something descriptive here and
## continues between newlines until another annotation.
## @tutorial https://website.com/tut
var speed: int = 100

Edit: Also, in a scenario such as this, it would be easier on the eyes if doc comments had a slightly different syntax highlight color than regular comments. Also helpful would be if doc annotations within doc comments were additionally highlighted

Edit 2: This article has an example of syntax highlighting for PHPDoc.

[YuriSizov]

Handling indentations on continued lines correctly also helps with visibility, like in this case:

## @description Blah blah something descriptive here and
##              continues between newlines until another doc annotation.

[ThakeeNathees]

@pycbouh currently I've planed to strip edges unless it's inside a [codeblock] and the first lines of comments starting with "##_" before any variable, functions, subclasses, constants, signals are considered class description and they are treated like this. any suggestions?

-------------------------------------
## this treated as brief description of the class
## as it doesn't have any @ annotation
-------------------------------------
## @brief: this is also brief description
## @desc: now this is description and
##        the edges are striped to make it as a whole paragraph
## @tutorial: http://www.mysite.com/tutorial/
-------------------------------------
## this is brief description
## @tutorial: http://www.mysite.com/tutorial/
## so this must be description
-------------------------------------
## @desc: the description
## @tutorial: http://www.mysite.com/tutorial/
## so this must be brief description
-------------------------------------
## @brief: brief description
## @desc: description
## @tutorial: http://www.mysite.com/tutorial/
## this line is not a part of the documentation <-----------

[YuriSizov]

@ThakeeNathees I think implicit description should only be generated based on commented lines before any @ keywords are used. So examples 3 and 4 are just as invalid as the 5th. Less ambiguity this way.

I think we need to drop either @desc or @brief in favor of fully relying on implicit descriptions. Probably should make implicit lines be @brief, while a full @description field can be used for a more detailed documentation, if one is required. From my experience it would be natural to do something like this:

## Get all character data from a string
##
## @usage: get_chars("My example string")
## @description: For an arbitrary string value generate a complete set of character data.
##               Returned value is an array of dictionaries, following this structure:
##               [codeblock]
##               {
##                 "character": "M",
##                 "utf8_code": "004D",
##               }
##               [/codeblock]
## @seealso: [method get_chars_unicode]
func get_chars(value : String) -> Array:
  pass

[aaronfranke]

@pycbouh That is alignment, not indentation, since it depends on each extra line being offset the same number of characters to match the first one. I think it's much better to do this, since it doesn't depend on the user's tab width but still allows tabs:

## @description
##  Blah blah something descriptive here and
##  continues between newlines until another doc annotation.

[YuriSizov]

I don't think that handling both cases is mutually exclusive. 🙂

[Zylann]

Just thought a bit about the script-wide doc block, and I think it can simply be solved by docs being separated by a line.

If a script contains this:

## @description This is the main class

class SubClass: # Happens to be undocumented
    # ...

or

## @description This is the main class

## @description This is the sub class
class SubClass:
    # ...

Because at least one space (or other non-doc-able symbol, like extends or class_name) separates the doc and whatever symbol that follows, we can fairly assume it's not about the subclass, but the script itself. So the defining condition would be, being before any doc-able symbol, and not directly above one.

[ThakeeNathees]

the main class description is inside the class block itself, IMO subclass doc should be like that

extends Node

## the brief description of the main class
## @desc: and the description of the main class
##        they are inside the class

class SubClass:

    ## brief description of the sub class
    ## @desc: similarly for sub class descriptions
    ##        should go inside it's class

    ## but other properties description
    ## will be above the declaration
    var x

@Zylann

[Zylann]

@ThakeeNathees but that doesn't solve the problem, because you have the same situation between the class description and the var in you snippet. So it should still be possible to have docs be just above the symbol they are documenting.

[ThakeeNathees]

@Zylann Sorry didn't get it. every documentation of a property should be right above it without any spaces and a big block of comments in the beginning of the file applies to the class (brief description, and the description). But if it touches a variable (no empty line in-between), then the description applies to the variable instead.

[Zylann]

@ThakeeNathees so since line separation is handled, that means we don't have to put docs inside sub-classes, they can be on top like everything else. The only natural exception is the script itself, where it can be at the very top.

[ThakeeNathees]

every class doc (including main script class and it's subclasses) can omitted if it's unnecessary, but to have them they must be written before all variable, functions, subclasses (and if it's a sub class then it's subclasses), constants, signals but the class_name doesn't considered since it's optional and it need an empty line after that ( or except class_name, non-doc comment are ok)

[Zylann]

What I mean is that since we can handle ambiguity with the main script using one or more lines of separation, I'd prefer subclass docs to be above them, not inside. I find it easier on the eyes to see them indented with the class name rather than with its members.

## the brief description of the main class

extends Node

## brief description of the sub class
## @desc: longer description of the sub class,
##        which can be on top since it can be des-ambiguified
##        with empty lines.
class SubClass:
    ## Member docs can be here
    var x
    ## And function docs too
    func foo():
        pass

extends Node

## @desc If the main script is not documented,
##       it's still not ambiguous as long as this doc
##       is right above its documented symbol.
class SubClass:
    pass

## @desc If the subclass isn't documented,
##       description of the main script only has to be separated 
##       from it by at least one line, which doesn't have to be an empty one.
##       (not saying this example is a good idea anyways)
extends Node
class SubClass:
    pass

Also if you really need some breathing space cuz the doc is long (sometimes I do, rarely^^), you can use extra ## to keep it as the same block:

## Brief description
##
## @desc Long description with code and schemas and whatnot,
## which can span many lines with paragraphs and maybe top-notch ASCII-art
## to show a visual concept.
##
## o----o   y
## |    |   |
## o----o   o--x
##
## Some footnotes over
## multiple lines, and possible trailing line
## 
class SubClass:

[ThakeeNathees]

@Zylann yeah, that's possible too, I should add an exception for sub classes, still need others opinion on subclass doc inside or just above it.

[ThakeeNathees]

@Zylann your examples looks easier to catch I'm going to implement that, if any one have suggestion let me know.

[hilfazer]

@Zylann yeah, that's possible too, I should add an exception for sub classes, still need others opinion on subclass doc inside or just above it.

Above.

Otherwise there will be 2 consecutive doc blocks - first for the class and second for first class member.

Edit: whoops, i just noticed you said "subclass" and my opinion was about inner class.

[NathanLovato]

I agree with having doc-comments always follow the same position rule, and having them always above has the advantage of working for everything.

We've had to try different syntaxes when writing gdscript docs maker, and I found it really helpful to have doc-comments that way. You open a class you don't know: the first line explains its purpose.

It's cool to see this happening, and the syntax looks good.

[ThakeeNathees]

according to the GDScript naming convention

Prepend a single underscore (_) to virtual methods functions the user must override, private functions, and private variables:

I should exclude the private functions, and variables (with underscore prefix) right? (if so does it applies to signal names, constants, how about class names starts with (_))

[YuriSizov]

Well, virtual methods shouldn’t be excluded, but this convention doesn’t separate them from private methods that probably should be excluded.

[ThakeeNathees]

@pycbouh how about

## @private:
var _internal

[YuriSizov]

@ThakeeNathees That's certainly what languages like JavaScript would do, but that's like one step away from @private annotation, so I'm not sure. Maybe someone else has a good idea.

[Zylann]

Maybe I'd not make it opt-out like ## @private just for docs to ignore it (too many places to add for no real benefit), instead I'd make that opt-in, ## @virtual or ## @protected (or as annotation, either way. Could be useful to code completion).

[ThakeeNathees]

we cant use annotation to denote private doc, which deceives the user that it really a private or protected property unless GDScript support those features.

[Zylann]

virtual is a thing in the doc, and found as this in the engine: https://github.com/godotengine/godot/blob/945ed76adb2bf54b1535f03047f642daa79f2cff/scene/gui/base_button.cpp#L397 Probably has no real meaning in GDScript functions though, which are all virtual de facto.

Another way would simply be to ignore _ methods which aren't documented.

[aaronfranke]

What about this?

• Allow placing documentation on any method, even _ methods, and generate docs for them.

• If the method does not have docs, and it starts with an underscore, hide it from generated docs.

Private methods don't need docs, only having comments works fine for them. Yet, if there is a virtual method which users want to add documentation to, they should be able to.

What I'm not sure about is what to do with variables that start with _. Maybe the above should be applied to variables too, or maybe it's better to always ignore them?

[Zylann]

What I'm not sure about is what to do with variables that start with _. Maybe the above should be applied to variables too, or maybe it's better to always ignore them?

Same as functions, I'd say

[NathanLovato]

I think it's worth asking @vnen: is there any plan to have some keyword to mark private and/or virtual functions?

---

In gdscript docs maker, I've added annotations for these under the form of tags (virtual, abstract) so we have some indication that a function is virtual or a class is meant to be extended and not instanced. For example, in our steering-based AI framework: https://www.gdquest.com/docs/godot-steering-ai-framework/reference/classes/gsaiproximity/.

Before I found that having pseudo-private variables and functions was enough, but now we make tools and APIs, having the same convention for virtual and private isn't ideal. It can also be confusing for beginners, from my experience teaching Godot and GDScript.

[dark-penguin]

Using string literal is easier to implement and would work for function and maybe for classes (even no way to distinct between description, brief_description, tutorials) but what to do with member variables, signals, enums, constants ?? using docstring for functions and something else for enums brings inconsistency, (if you have any idea on how to resolve this, a mock up pice of code would be helpfull).

Why can't we do the same thing? (Am I not getting something really obvious?)

var maxspeed = 100
""" Maximum player speed """

var minspeed = 100
""" Minimum player speed """

This even plays well with comments. Comments usually go above, and documentation usually goes below:

# Actually, you can go faster than this with enough medications, but we ignore this here
var maxspeed = 100
""" Maximum player speed """

var minspeed = 10 # Should be low enough
""" Minimum player speed """

# Default speed for a average healthy human - don't rely on this too much
var speed = 50
"""
Default player speed

It could vary a lot, from [minspeed] all the way up to [maxspeed],
so don't rely on this too much.

min: 10
max: 100
"""
# TODO: this documentation entry is worded poorly - redo this

(With highlighting, this would look even better.) (EDIT: added highlighting)

I would argue that "there is only one, and obvious, way to do it, and that's docstrings".

[dark-penguin]

no way to distinct between description, brief_description, tutorials

The first line is brief description. The rest of it is full description. I don't see any existing built-in documentation where "brief description" would require multiple lines, or where "tutorials" would mean more than a single-line link, so Tutorials: docs.godotengine.org in the last line should be fine. (Meaning, if you really need a fancy caption for the tutorials, and you can't use markup, then just see if the last line begins with the word "tutorial" case-insensitive, and has an empty line before it.)

It can support some kind of markup, but remain readable: if you want a heavily stylized documentation, make a separate documentation file - docstrings are just for "simple" inline documentation. Things like @param: or @tutorial inside the docstring are also fine, but they must not duplicate anything that is possible to deduce from existing code (like type hints).

func frobnicate(something):
    """
    Let the frobnication begin

    This function is tricky. To understand what it does, you have to read the tutorials and
    experiment with it yourself.

    But here are a few examples:
    ```frobnicate(5)``` - does a foobar-type frobnication on an integer (which could also be a string)
    ```frobnicate("5")``` - does a completely different thing, for those coming from Javascript

    ### Tutorials: http://frobnicate.org/getting-started
    """

[NathanLovato]

@dark-penguin Thinking in terms of docstrings, coming from Python, is fine, but other languages use other conventions that work very well too, sometimes called doc-comments. That @ in comments is inspired by the JSDoc notation.

Godot already supports getting information from comments through the language server, and it already extracts as much information as it can about the code. The @ is there to write things the server cannot get. Also, having @ means you have a unique token to parse, keeping the implementation as simple as can be. With different notations like ###, Tutorials:, and backticks for code blocks, you complicate the code (NB: Godot doesn't come with a markdown parser).

[dark-penguin]

I think GDScript should try its best to stay as close to Python as possible. "Other languages" means only C# (and maybe later at some point something else?), and the C# implementation should stay as close as possible to C#; there is already a whole other language parser for C#, so it would be natural to have a separate documentation parser for it. I'd say it's better to have separate implementations for each language rather than forcing both languages to use syntax that's not native to either of them. For C#, apparently that means you can even use extraction from comments, which means that the C# parser will be easier to implement.

But for GDScript, docstrings are definitely the way to go. Think about it this way: we can add an inter-language syntax for all the "other languages", but I'd say for GDScript it's well worth to implement docstrings as well, because it's the primary one, and so it deserves best support. And if so, then we might as well remove the "inter-language" one from GDScript and leave it for "other languages" only.

I don't mind any notation for inside docstrings (except xml; @something looks fine). Trying to answer "why not docstrings?", all I saw was that post about the only obstacle being "but then how do we get the tutorials section?". And that looks like a complete non-issue; you can do that any way you want! Do @tutorials, JSDoc-style; or just parse the bottom line, bash-style; or add a token for captions - and maybe another token for monospace, and that's all the markup we need. To me, it seems that reinventing docstrings with annotations for sake of "other languages" is way more trouble than just writing a native implementation for every language's documentation style. Also, I just can't agree with putting the docstring before the definition. They are supposed to go after!..

EDIT: My biggest gripe is functionality collision. Comments are for commenting, annotations are a part of code, and docstrings are for documentation. Now we want to put them together in one big mess, when a beginner can't even tell if annotations are code or basically comments, and comments are sometimes documentation. Those three things should be different, and very obviously different.

[NathanLovato]

I think GDScript should try its best to stay as close to Python as possible

Except for the fact they're indent-based, GDScript and Python are very different languages. They have almost nothing in common.

Taking inspiration from the good parts of Python that make sense in the context of game programming and Godot, sure, why not. But they shouldn't stay as close as possible for the sake of it, and they aren't close in the first place.

for GDScript, docstrings are definitely the way to go

GDScript doesn't have docstrings, but it has doc-comments already. Doc-comments being the norm in many other languages like JavaScript, C++, Go, etc. That's what I meant by:

other languages use other conventions that work very well too

Now we want to put them together in one big mess,

Let's be concrete. We're just talking using comments, or lines starting with ## vs multiline strings, i.e. """""". I don't see much difference between the two examples below, especially with syntax highlighting:

# Base type for group-based steering behaviors.
# @category - Base types
class_name GSAIGroupBehavior
extends GSAISteeringBehavior

# Container to find neighbors of the agent and calculate group behavior.
var proximity: GSAIProximity

var _callback := funcref(self, "_report_neighbor")

func _init(agent: GSAISteeringAgent, _proximity: GSAIProximity).(agent):
    self.proximity = _proximity

# Internal callback for the behavior to define whether or not a member is
# relevant
# @tags - virtual
func _report_neighbor(_neighbor: GSAISteeringAgent):
    return false

Note this is docs maker syntax and not the final syntax for this task, but this is for the sake of example. With multiline strings:

class_name GSAIGroupBehavior
extends GSAISteeringBehavior
"""Base type for group-based steering behaviors.
@category - Base types"""

var proximity: GSAIProximity
"""Container to find neighbors of the agent and calculate group behavior."""

var _callback := funcref(self, "_report_neighbor")

func _init(agent: GSAISteeringAgent, _proximity: GSAIProximity).(agent):
    self.proximity = _proximity

func _report_neighbor(_neighbor: GSAISteeringAgent):
    """Internal callback for the behavior to define whether or not a member is
    relevant
    @caegory - virtual"""
    return false

[NathanLovato]

However, this, to me, looks weird. And I'd bet on that confusing users more than just having comments everywhere.

# Actually, you can go faster than this with enough medications, but we ignore this here
var maxspeed = 100
""" Maximum player speed """

[dark-penguin]

We are talking about few different things, I think. Let's separate them into different points.

The first problem I see here is that in the first example, those things are comments. Meaning, they should be completely ignored by anything. In the second example, the docstrings will be highlighted yellow, so they are obviously not comments. I think that is much better.

The second problem is, the first example actually looks pretty good from this point of view: "Let's make a small dev tool that shows our short comments in popups". But it no longer looks good from this point of view: "Let's write extensive multi-line documentation, possibly with some markup". Just rephrase your comments as what you would want to see in documentation, add brief and full descriptions, and links to the tutorials. I will look like "code with more comments than code", which is kind of repulsive on its own. A code with large docstrings, however, looks much better.

The third thing is, I don't have anything against @category notation in docstrings, but the end of the first example looks like "Have I commented an annotation?.. Ah. no, it's not an annotation...". It does not look like a commented annotation in a docstring, especially if you put the """-s on separate lines.

About my "weird looking" example: you usually do not need to have both a comment and a docstring for one variable declaration. But with my syntax, you can. (With yours, you can not. Or at best it will not be obvious where the comment ends and the docstring starts.) Because docstrings are "production code"! They are not comments!