Plugin documentation generation + builtin rss plugin

[Andrew-Morozko]

Implements #128

A couple of diviation from the spec: config options are represented like so:

basic_auth {
    username="username"
    password="password"
}

not

basic_auth_username = "username"
basic_auth_password = "password"

This seemed nicer to look at, but also it allows for basic_auth to be an optional block, but username and password are both required if the basic_auth block is present. In the original spec you could've specified basic_auth_username without a password, for example.

Also, in addition to pub_date as text I provide pub_timestamp (a unix-timestamp of pub_date) on a best effort basis (because sometimes pub_date can't be parsed)

[Andrew-Morozko]

@dobarx

My plugin fails TestAllPluginSchemaValidity/blackstork/builtin/rss test, presumably here because I'm using embedded block inside of a config block.

It works perfectly fine in the binary. Is it ok to loosen the restrictions in this test, or is something in your code relies on this property?

[dobarx]

@dobarx

My plugin fails TestAllPluginSchemaValidity/blackstork/builtin/rss test, presumably here because I'm using embedded block inside of a config block.

It works perfectly fine in the binary. Is it ok to loosen the restrictions in this test, or is something in your code relies on this property?

I think it's Ok to use block, but then maybe update that test to allow including block types also. Also, check plugin doc generation templates just generate-docs. They probably do not expect blocks atm so it needs to be updated if you want to use block specs inside plugin configs.

[Andrew-Morozko]

Finally done with it. Some notes:

128 and #119 are done.

124 is done partially, some questions remain (@traut):

• [x] I don't see a reason to store json_schema. It is fully generated from plugin metadata; no need to store it in the plugin metadata as a blob.
• [x] There are no tags on plugins, but in the frontmatter of the plugin.md.gotempl there is a placeholder for tags. Tags on plugins: yay or nay?
• [x] I renamed all documentation strings to Doc instead of Description, does it matter?

This PR doesn't include the regenerated docs:

• It would be way too big
• All plugins except RSS lack the documentation, so current docs are pretty barren

After this is merged we should create a PR that adds a bunch of documentation.

P.S.: @dobarx we talked in DM about refactoring the code a bit more, but I'd rather do it in a separate PR. Also I very much enjoy the formatting of protobuf files, but buf has a builtin formatter, perhaps we should use it to be a bit more standardized? Dealing with whitespace in surrounding lines is a bit of a mess :/

Here's an example of generated docs for RSS:

---
title: rss
plugin:
  name: blackstork/builtin
  description: "Fetches an rss or atom feed"
  tags: ["rss","http"]
  version: "v0.4.1"
  source_github: "https://github.com/blackstork-io/fabric/tree/main/internal/builtin/"
resource:
  type: data-source
type: docs
---

{{< breadcrumbs 2 >}}

{{< plugin-resource-header "blackstork/builtin" "builtin" "v0.4.1" "rss" "data source" >}}

## Description
Fetches an rss or atom feed

The data source is built-in, which means it's a part of `fabric` binary. It's available out-of-the-box, no installation required.

## Configuration

The data source supports the following configuration parameters:

config "data" "rss" {
  # Authentication parameters used while accessing the rss source.
  #
  # Optional
  basic_auth {
    # Required. For example:
    username = "user@example.com"

    # Note: you can use function like "from_env()" to avoid storing credentials in plaintext
    #
    # Required. For example:
    password = "passwd"
  }
}

## Usage

The data source supports the following parameters in the data blocks:

data "rss" {
  # Required. For example:
  url = "https://www.elastic.co/security-labs/rss/feed.xml"
}

[traut]

I don't see a reason to store json_schema. It is fully generated from plugin metadata; no need to store it in the plugin metadata as a blob.

Sure, let's add it when we have a clearer use case!

There are no tags on plugins, but in the frontmatter of the plugin.md.gotempl there is a placeholder for tags. Tags on plugins: yay or nay?

It would be great to have the tags! It simplifies filtering/discovery in the docs for humans -- I'll add a search to the docs soon. On that note, could you please check if the code that generates plugins.json in docgen needs adjustments? It's also fine if you prefer to do that together with the docs PR

I renamed all documentation strings to Doc instead of Description, does it matter?

All fine with me!

Much appreciated, thank you, @Andrew-Morozko !

[Andrew-Morozko]

All done, ready to merge @dobarx