Enhancement: New field type for 'list' or array of sub-collections.

[bwklein]

With 'Page Sections' or 'Blocks' in front matter data, there needs to be a way to define section/block templates and then manage a list/array of them in front matter.

To add, remove, change order of elements in the list and edit the front matter data in the block elements.

For an example see: https://forestry.io/blog/sawmill-layout-composer-for-hugo-and-forestry/#hugo-example

For a working example of this system in action see: Website - https://psrfcu.org/ Repo - https://gitlab.com/psrfcu/psrfcu-2019/-/blob/master/content/_index.md

[estruyf]

Thanks @bwklein for the suggestion! The repo link you shared ends up on a 404 page. Would you be able to check the URL again? Once I can see how you are using it, I can understand the logic a bit better.

[bwklein]

Note, that with content files like this, there usually isn't any {{.Content}} in the body of the markdown file. Everything is defined in front matter variables to populate variables in the partials used to render the data into the page block.

It makes the site more like a traditional CMS where the information for the design components (blocks) is stored in the front matter data model and not all in markdown/html within the body of the content file.

[estruyf]

Thank you for providing a sample! This makes it clear to me. As of what I can see this is specific to Forestry. I like the idea, but also think it's a bit abusing the front matter. Shortcodes would be a cleaner way and probably makes it better suitable for other SSGs.

Just to understand a couple of things. These page sections, where do you define these? I assume these are custom things you need to configure before they will be able to be used. Is that true?

[bwklein]

Yes, essentially you make design elements or blocks in your partials directory. Then you have a layout type that reads through the front matter and for each 'page_section' it calls the appropriate 'block' and passes the data from that page_section array element into the partial. All of that would need to be setup first, to define what inputs are required for that block partial.

{{- define "main" -}}
  {{ $count := 0}}
  {{- range .Params.page_sections -}}
    {{ partial (printf "blocks/block-%s.html" .block) (dict "Section" . "Page" $ "section_count" $count ) }}
    {{ $count = (add $count 1) }}
  {{- end -}}
{{- end -}}

You also need to define the set of inputs for a particular block and use that for editing the data in the array of page_sections/blocks. In Forestry this is done with their Front Matter Templates, in Cloud Cannon this is done with array structures, see the attached file starting at line 80 for the Cloud Cannon array structures that match with the file I pasted above.

cloudcannon.toml.zip

This approach works great in the situation where you want to reuse design elements on different pages and the editor can move them around and fill in the required information for each block as they 'assemble' the page from top to bottom.

In the site I posted the file for above, there are 19 different blocks that someone could build a page out of.

You are correct shortcodes could work too, but would make it difficult to reopen the editor for the block to make changes to an existing shortcode, unless you had a parser that would read the shortcode string and populate the snippet insertion UI with the values from the shortcode string. It seems like it would be a bit easier to define an array field type and a mini 'content type' for each 'block' that could be added into the array.

[zivbk1]

I think of shortcodes as a simple way to insert a smaller design element (figure, table, video, tweet) into the body of a larger Markdown file. Where this 'blocks' system is a way of composing entire pages from top to bottom, where each layer in the stack has a specific set of inputs to that specific layer.

[zivbk1]

One thing to note is that shortcodes will NOT work within the content of a layer field. So you will see in my example file code that there are 'content_markdown' fields for some blocks. If you put a shortcode in there it won't render the shortcode result into the page, there will just be a shortcode string in the content of that block. You can 'markdownify' the content_markdown into HTML, but the shortcodes are ignored. There is an existing Hugo project issue to resolve this and allow for shortcodes in frontmatter text to be rendered.

[zivbk1]

@estruyf this has become a blocker for me. I can't use it on sites where I have arrays of data in Front Matter. Do you have any suggestions for a workaround, or ideas on how to implement this in FM?

[estruyf]

@zivbk1 added it to the board for the next release. At the moment there is no workaround for it. Think it is best to start with #197

[estruyf]

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

[bwklein]

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

I think it is closer, but the missing piece is a field type in a template where you can choose from a set of defined data structures to add them into an array in the front matter. Do the examples above help or should I setup a more simple example/test for this feature?

[apowell656]

My two cents - I use Forestry Blocks for a two client sites (Hugo) and one personal(Pelican) to allow the users a familiar front end to contribute content and build pages. Their "Blocks" feature is a game changer for changing pages. I thought about asking for this feature also, but I could not even think of how it would take advantage of the existing FM UI without a modal/dialog box and my Pelican sites don't use shortcodes, so I did not propose them.

If it can happen that would be cool if not I might just make project level snippets.

[estruyf]

Project level snippets is going to be the next major improvement for Front Matter, this is certain

There needs to be a benefit of using these over what VS Code provides by default. We're thinking about:

• allowing you to add snippets
• visualising all snippets on a dashboard
• dialog to insert placeholders, which include inputs/choices/...

One thing that would make it great, is to have a way to get easily update the snippets that are already defined in the content.

About the list field, I've been thinking of using the same data field schema (JSON schema) which is used in the new data view.

[wemasoe]

@estruyf yes that are nice features.

About the list field, I've been thinking of using the same data field schema (JSON schema) which is used in the new data view.

i am still not familiar with json and the scheme technics. Hopefully this project may clarify some aspects of using json and json schemes.

i am thinking of the Specifications that are announced on the Specification documents on the JSON Schema org.

Versioning and looking forward is nice, but if you are out of date it may be getting complicated to be up to date. telling it from a perspective of an unemployed who stucks learning after some issues. Which schema is the right one? Should this be hardcoded somehow, or would/should be let over to the developer?

i am getting overhelmed trying to find ressources for a clear explanation for json schemes, it looks realy fluent. but, it is a nice solution, but for me not clear enough right now.

also yaml is explaining itself as a clear solution, binding both together with a clean solution might be helpfull. yaml to json schema and json data, and also json schema and json data to yaml? decide :-) well i need some vacation :-)

[estruyf]

@wemasoe the version of the JSON schema does only matter when you are going to create really complicated data types. If you only need a couple of data fields, the schema is "pretty" easy to understand. There is always a trade-off with each road picked or decision.

The same goes for YAML, it is easy to understand, but this is just the data representation. When you would create a YAML schema to define the data structure, it would also mean you need to think it through a bit and do some trial and error.

Another reason why JSON Schema has been taken is that VS Code its settings are JSON driven and this keeps it all aligned.

Future

If we get more contributors on board, or if I manage to make more time available, one of the ideas that are on the backlog is to create a content type/settings dashboard (#129). This could become a drag and drop kind functionality to create data types and content types. Which eventually might make it easier for any that is not familiar with it.

[wemasoe]

@wemasoe the version of the JSON schema does only matter when you are going to create really complicated data types. If you only need a couple of data fields, the schema is "pretty" easy to understand. There is always a trade-off with each road picked or decision.

The same goes for YAML, it is easy to understand, but this is just the data representation. When you would create a YAML schema to define the data structure, it would also mean you need to think it through a bit and do some trial and error.

Another reason why JSON Schema has been taken is that VS Code its settings are JSON driven and this keeps it all aligned.

Future

If we get more contributors on board, or if I manage to make more time available, one of the ideas that are on the backlog is to create a content type/settings dashboard (#129). This could become a drag and drop kind functionality to create data types and content types. Which eventually might make it easier for any that is not familiar with it.

step by step, everything is going to be good

i have to say the vscode extension Front Matter is a nice project, again, again, again.

[zivbk1]

@zivbk1 does #197 get this supported as well? Or are there missing pieces still?

I think it is closer, but the missing piece is a field type in a template where you can choose from a set of defined data structures to add them into an array in the front matter. Do the examples above help or should I setup a more simple example/test for this feature?

@estruyf does this help clarify the need or should I provide a better example?

I have begun setting up a Hugo starter that uses this method of page building. https://gitlab.com/zivbk1/hugostarter

You can see in that project that there are already 2 'blocks' that a person can use to build a page. https://gitlab.com/zivbk1/hugostarter/-/tree/main/layouts/partials/blocks

You can see here the code that is used to take the frontmatter of a page and call the correct block partial into place from top to bottom of the page. https://gitlab.com/zivbk1/hugostarter/-/blob/main/layouts/_default/page.html

Finally, you can see in this file where both blocks are called into the page from the page_sections array. https://gitlab.com/zivbk1/hugostarter/-/blob/main/content/_index.md

Here is an example of the definition of that 'page' content type. Where there are two choices (for brevity) that can be 'added' into the page_sections front matter array of a 'page' content type.

"frontMatter.taxonomy.contentTypes": [
  {
    "name": "page",
    "fileType": "md",
    "fields": [
      {
        "title": "Title",
        "name": "title",
        "type": "string"
      },
      {
        "title": "Publishing date",
        "name": "date",
        "type": "datetime"
      },
      {
        "title": "Article preview",
        "name": "featured_image",
        "type": "image",
        "isPreviewImage": true
      },
      {
        "title": "Is in draft",
        "name": "draft",
        "type": "draft"
      },
      {
        "title": "Page Sections",
        "name": "page_sections",
        "type": "choice",
        "multiple": true,
        "choices": [
          {
            "icon": "hail",
            "title": "Hero Banner",
            "fields": [
              {
                "title": "Page Block",
                "name": "block",
                "type": "string",
                "hidden": true,
                "default": "hero"
              },
              {
                "title": "Hero Text",
                "name": "heading",
                "type": "string"
              },
              {
                "title": "Background Color",
                "name": "background_color",
                "type": "choice",
                "choices": [
                  {
                    "id": "white",
                    "title": "White"
                  },
                  {
                    "id": "green",
                    "title": "Green"
                  },
                  {
                    "id": "black",
                    "title": "Black"
                  }
                ]
              },
              {
                "title": "Text Color",
                "name": "text_color",
                "type": "choice",
                "choices": [
                  {
                    "id": "white",
                    "title": "White"
                  },
                  {
                    "id": "green",
                    "title": "Green"
                  },
                  {
                    "id": "black",
                    "title": "Black"
                  }
                ]
              }
            ]
          },
          {
            "icon": "newspaper",
            "title": "Markdown Content",
            "fields": [
              {
                "title": "Page Block",
                "name": "block",
                "type": "string",
                "hidden": true,
                "default": "content"
              }
            ]
          }
        ]
      }
    ]
  }
]

In this context, any 'icon' defined would be pulled from something like https://fonts.google.com/icons?selected=Material+Icons

After building that data structure, it seems like it would be nice to define choice arrays somewhere else in the config file that can be reused by referring to them by name. Like a background color and text color set of options that can be used in different choice fields to keep things DRY.

[zivbk1]

To add to my comment above. It would be nice to see the list of page sections in the UI in an expandable accordion or something like that, where the order of the array can be changed and then edited in the expanded accordion space below the heading or maybe an editing modal that opens for that specific data item in the array. It would also be good to identify what field in the block you want to show in the stacked list of page_sections, similar to "labelField" in the data file definition. This is where it would be nice to just show the icon and then the text for the "labelField".

A lot of what I am talking about seems very similar to how the data file editor works now. Maybe use a similar UI with the stack of sections similar to the array of data items in a data file, and a very similar order and editing UI for those sections.

[estruyf]

Still work in progress, but this is what the new collection data field looks like:

[estruyf]

Style changes + some fixes have been added. Just committed the first version of the new control.

In order to use it, you'll need to define the field in your content type as follows:

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "data-collection",
  "dataType": "page_sections2"
}

In the frontMatter.data.types, you can define your data type to use for the collection.

[zivbk1]

Can "dataType": "page_sections2" be set up as a list of data types that can be added into the Record set?

In most use cases there would be different 'blocks' or data structures that would be added to the list of records. Where in your example 'Record 1' would be one data type, 'Record 2' would be another, and so on.

Also, would there be an ability to reorder the list, to put Record 3 above Record 1?

This is some example frontmatter that I would want to make work.

title: Hello World
date: 2022-02-04T21:48:46.000Z
draft: true
layout: page
page_sections:
  - block: hero
    heading_markdown: Hello World
    background_choice: gray
    text_choice: white
  - block: content
type: page
lastmod: '2022-02-04T23:52:45.963Z'

Where the first 'block' or record in the array has a data structure that includes the fields, block, heading_markdown, background_choice, text_choice and a second data type that only contains the field block set to the default value of 'content'.

[estruyf]

1. If it can be defined in a JSON schema, it will be possible, but having different data types, loaded in the data collection field is not “yet” possible.

2. Technically it’s possible, I first focused on the main collection functionality and getting the fields right

[estruyf]

Been thinking, so if the dataType can contain multiple data types, we could for instance show first a dropdown that allows you to select the type. Once you choose the type, it will show the additional data type's fields.

In order for this to work, we need to have an id field that corresponds to the data type you picked. Otherwise, it would be guessing which one you took. In your case, it is block, could we change it to type? That way it would be a bit more generic.

[apowell656]

FWIW - In my forestry setups I use template and it works, so I don't think using type will be an issue. A generic term is probably the best way to go.

[estruyf]

Thanks for the feedback! Have been coding during my flight. Will commit the changes later.

Changed it to data blocks and block types. When configuring, you can select one type or many. Once you pick many, you'll need select the type to use and add the data.

Hope it will make sense.

[zivbk1]

Changed it to data blocks and block types. When configuring, you can select one type or many. Once you pick many, you'll need select the type to use and add the data.

This makes perfect sense to me.

[estruyf]

The field will need to be configured as follows:

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "block",
  "blockType": "type1"
}

or

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "block",
  "blockType": ["type1", "type2"]
}

[zivbk1]

The field will need to be configured as follows:

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "block",
  "blockType": "type1"
}

or

{
  "title": "Page sections",
  "name": "page_sections",
  "type": "block",
  "blockType": ["type1", "type2"]
}

I'm going to try it today! 🙂

[apowell656]

I'm trying it out-ish, but I think I need a primer. :-(

[estruyf]

@apowell656 you will need two things:

1. A data type defined in the frontMatter.data.types settings
2. A content type with the block field defined

frontMatter.data.types - setting

"frontMatter.data.types": [
  {
    "id": "test2",
    "schema": {
      "type": "object",
      "required": [
        "name",
        "url"
      ],
      "properties": {
        "name": {
          "type": "string",
          "title": "Name"
        },
        "url": {
          "type": "string",
          "title": "URL"
        }
      }
    }
  },
  {
    "id": "test3",
    "schema": {
      "type": "object",
      "properties": {
        "block": {
          "type": "string",
          "title": "Block"
        },
        "name": {
          "type": "string",
          "title": "Name"
        }
      }
    }
  }
]

frontMatter.taxonomy.contentTypes - setting

"frontMatter.taxonomy.contentTypes": [
  {
    "name": "page",
    "fields": [
      {
        "title": "Page sections",
        "name": "page_sections",
        "type": "block",
        "blockType": ["test2", "test3"]
      },
      {
        "title": "Is in draft",
        "name": "draft",
        "type": "draft"
      },
      {
        "title": "Title",
        "name": "title",
        "type": "string"
      },
      ...
    ]
  }
]

[apowell656]

@estruyf that put me on the right track. I looked into the documentation to get me across the finish line, but I think I am missing something. Do I need to create a data.type for each "block"/template? _I tried changing the template to block in the pagesections with the same result.

[estruyf]

@apowell656 yes, you'll have to specify what your block looks like, otherwise, it is a bit hard to render the form.

To compare it to Forestry:

• The data type you define is the repeatable field;
• The block field you define in the content type, is like the block field you define in Forestry, but instead of liking a field(s) to it, you like a data type to it.

[zivbk1]

I am getting an "Error loading field" error when trying to use the 'choice' field type in a block the data type definition.

"frontMatter.data.types": [
    {
      "id": "hero",
      "schema": {
        "type": "object",
        "required": [
          "heading_markdown"
        ],
        "properties": {
          "heading_markdown": {
            "type": "string",
            "title": "Heading"
          },
          "background_choice": {
            "title": "Background Color",
            "type": "choice",
            "choices": [
              { "id": "black", "title": "Black" },
              { "id": "psrfcu", "title": "Blue" },
              { "id": "gray", "title": "Gray" }
            ]
          },
          "text_choice": {
            "type": "string",
            "title": "Text Color"
          },
          "blockType": {
            "type": "string",
            "title": "Block Template"
          }
        }
      }
    },
    {
      "id": "content",
      "schema": {
        "type": "object",
        "properties": {
          "blockType": {
            "type": "string",
            "title": "Block Template"
          }
        }
      }
    }
  ]

[zivbk1]

Also, is it possible to use the "default" and "hidden" properties for a particular field?

For example, to always set blockType: hero in a page_section block.

"blockType": {
  "type": "string",
  "title": "Block Template",
  "default": "hero",
  "hidden": true
}

I don't want the editor to change that field and break the page.

NOTE: I get "Error loading field" now if I add these.

[estruyf]

I am getting an "Error loading field" error when trying to use the 'choice' field type in a block the data type definition.

@zivbk1 that is because it is a JSON schema and the types need to be valid JSON known types: string, number, boolean, object, array. Custom types are not "yet" supported. It is doable to create custom types, but it means that there need to be custom renderers created for each new type. We probably need image and DateTime support as well.

In case of the choices, you can use the following:

{
      "title": "Background Color",
      "type": "choice",
      "enum": ["Black", "Blue", "Gray"]
}

[zivbk1]

It might be good to not show the 'blockType' field in a page_section at all. I can't think of a use case where someone would want to edit that. If you want to change the blockType, then just add another block to the list of the correct type and delete the current one that is the wrong type.

[zivbk1]

We probably need image and DateTime support as well.

I was just about to ask for that. 😄

[zivbk1]

Basically, anything you can do in the frontmatter UI, you could do within a block editor. Same field types, UI and functions.

[estruyf]

Also, is it possible to use the "default" and "hidden" properties for a particular field?

Yes, this is supported, but you should not be doing it for the blockType field, as that is automatically done by the extension.

It might be good to not show the 'blockType' field in a page_section at all. I can't think of a use case where someone would want to edit that. If you want to change the blockType, then just add another block to the list of the correct type and delete the current one that is the wrong type.

So, once a block has been created, you would only be able to move it or delete it?

[apowell656]

Okay, I like my tests. Final question is it possible to nest an array? I have a template that has nested paragraphs (about-section) I am mentally halfway paying attention D@&n Super Bowl.

[estruyf]

Basically, anything you can do in the frontmatter UI, you could do within a block editor. Same field types, UI and functions.

That is indeed what is needed. Also thinking about another option to extend the field field type. Right now it supports to only specified sub-fields, but what if we extend the field to allow field-bundle choices.

Maybe this block field, would better become a collection field, like how I initially had it in mind. As it allows you to reuse the data types from your data files/folders.

The block field, becomes a field type that allows you to do the same as what is defined above, but instead of specifying a data type, you specify a field block.

field block would be a new setting, that comes with an ID and fields (all the fields from Front Matter are supported).

This might be the best option after all. I might have been overthinking it due to the discussion we had about the file data/folders previously.

[zivbk1]

I actually love this new direction and it is closer to what I originally had in mind too.

I really just want a way to define 'blocks' (a configured selection of fields) and then have a way of picking from the defined set of blocks and adding them into an array ('page_sections' as an example of the array/collection name).

Once in the array, I can edit, move its position in the collection and delete them from the collection.

[zivbk1]

Also, it would be nice to specify a field in the field block to use as the text for the item in the list of blocks that have been added to the array. So the 'Title' of the field_block might be 'Hero Banner' as the label for selecting the correct block from the list. Then there is a 'heading' field in the block that you would like to see in the label for the block in the collection.

This way you can determine which 'hero' in the page_sections collection is the one that has the text you want to edit.

[zivbk1]

So, once a block has been created, you would only be able to move it or delete it?

Yes, you can edit the contents of the block, change the order in the collection and delete it. I just don't want to be able to modify the block identification field (whatever it is called). Because that field is used to determine the 'field_block' template to use and determine the Hugo partial to render that section in the page. All of the other fields in the block are editable.

[apowell656]

All - would anyone be willing to share a sample of usage?

[zivbk1]

All - would anyone be willing to share a sample of usage?

Here is a super basic demo. Where the main index page uses blocks.

https://gitlab.com/zivbk1/hugostarter

[estruyf]

@apowell656 @zivbk1 the block field will now be the json field.

In the example of @zivbk1 - https://gitlab.com/zivbk1/hugostarter/-/blob/main/frontmatter.json#L55-63

{
  "title": "Page Sections",
  "name": "page_sections",
  "type": "block",
  "blockType": [
    "hero",
    "content"
  ]
}

You'll have to change type to json, and blockType to dataType.

Currently, the new block field is in development.

[zivbk1]

@estruyf thank you. I'll update this demo project when blocks are ready. What use case do you anticipate for the json type?

[apowell656]

@estruyf and @zivbk1 thanks for links. I have validated my JSON to use nested content (my fields without nested content are working well), but the result is the error Error loading field1. The documentation is not fleshed out well enough to explain if I can do this fordata.types` is that the case.

[estruyf]

@estruyf thank you. I'll update this demo project when blocks are ready. What use case do you anticipate for the json type?

Just there in case you want to reuse the data types you already have defined. As it was already developed, I didn't want to remove it.

[estruyf]

@estruyf and @zivbk1 thanks for links. I have validated my JSON to use nested content (my fields without nested content are working well), but the result is the error Error loading field1. The documentation is not fleshed out well enough to explain if I can do this fordata.types` is that the case.

This has probably to do with an invalid JSON schema. Once the new block field is ready, I hope it will make it easier.