[RFE]: --format go template fields documentation

Luap99 commented 1 year ago

/kind feature

This is an continuation of my comment in https://github.com/containers/podman/pull/17443#discussion_r1101623219

Problem statement

I think we can all agree that most --format field names should be documented. @edsantiago did some great progress with his PR. However I see some problems with that approach as mentioned in the comment.

The main concern is around differences between docs and auto completion, right now the auto completion shows a bit to much, some nested internal struct are exposed which means that changing them can causes breaking changes to templates. Thus we do want them documented but also not shown on auto complete.

Also the current approach requires somebody to add descriptives comments for each field in the docs. This can be a lot of work. Most fields are actually already commented reasonably well via standard go docs in the code so there is a lot of copy paste involved.

Proposed solution

With this in mind I propose we need a custom tool that can write us the docs bases on our code comments. Go has strong AST parsing libraries (https://pkg.go.dev/golang.org/x/tools/go/packages and https://pkg.go.dev/go/ast) that can do the heavy lifting.

The way I image it working is to first search for all common.AutocompleteFormat(...) calls, this what the completion logic is based on. Then we can search the ast for our type that was given as argument, this struct contains all fields that can be used in the template (only exported ones are used). From there we can read each field and the go comment attached to it. This must happen recursively for all structs within this struct and also work for embedded structs.

To make field not show up it should be private or if that is not possible I suggest we add a new field tag called completion:"none":

type MyStruct struct {
    Name string
    Hidden string `completion:"none"`
}

The reason we use a tag and not a comment is because the completion code which uses reflect can read tags but not comment at runtime. This allows us to keep them hidden in docs and completion so they are always in sync without extra work.

Once we have the info we will then write it the stdout or file in the same tabular format that is used and the moment in the man pages.

| **Placeholder**        | **Description**                                                           |
|------------------------|---------------------------------------------------------------------------|
| .Field                 | Human description                                                         |
...

The man page validation script can then use this to diff against the man page, I am also open to different formats when it should make things simpler to compare.

Possible problems

This depends on the correctness of our go comments. Generally speaking I think most of them are good enough but some will require tweaking. Also stuff like FIXMEs should not be displayed, we likely need a way to exclude a certain part of comments that should not be part of public docs.
I have no idea how complicated the AST parsing really is. It could be more complex than I think.

github-actions[bot] commented 1 year ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 1 year ago

@Luap99 Is this still an open issue? Most of these are documented now. Just need a way to test for missing documents.

edsantiago commented 1 year ago

This is still very much an issue:

'podman image inspect': --format options ... are not documented in podman-image-inspect.1.md
'podman machine info': --format options ... are not documented in podman-machine-info.1.md
'podman machine inspect': --format options ... are not documented in podman-machine-inspect.1.md
'podman secret ls': --format options ... are not documented in podman-secret-ls.1.md
'podman system df': --format options ... are not documented in podman-system-df.1.md
'podman volume inspect': --format options ... are not documented in podman-volume-inspect.1.md
'podman volume ls': --format options ... are not documented in podman-volume-ls.1.md

Plus, this issue is about much more: as I read it, it's about autodocumenting (not just validating) and it includes the possibility of really getting rid of the unwanted options.

github-actions[bot] commented 1 year ago

A friendly reminder that this issue had no activity for 30 days.

saper commented 11 months ago

I think the actual data returned depend on the underlying data structure (image, container, etc.) that should be defined somewhere else.

But I came here looking for go functions available in the templates. For example, I can do this now:

$ podman container inspect --format '{{range $i, $env := .Config.Env}}{{ println $env }}{{ end }}' $container
PATH=/me/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TERM=xterm
container=podman
HOME=/me
HOSTNAME=9402498241ff

I can filter on the exact value:

$ podman container inspect --format '{{range $i, $env := .Config.Env}}{{ if eq $env "HOME=/me" }}{{ $env }}{{ end }}{{ end }}' $container
HOME=/me

but I do not seem to be able to search on a substring or invoke anything more complicated than a simple eq.

(I do not understand how call is supposed to work, https://pkg.go.dev/text/template#hdr-Arguments is very confusing).

Luap99 commented 11 months ago

@saper This issue is explicitly about generating the field doc description based on the actual structs used and not for general docs on how to use the template syntax. For that there is already https://github.com/containers/podman/issues/17676 so please follow that. Templates are limited to some extend and you cannot just call arbitrary functions in it. If you need more logic I suggest you use shell script.

saper commented 11 months ago

@Luap99 thanks for the pointer, I'll ask there.

containers / podman