Feature request: Add `hidden` property for frontmatter

[tuyen-at-work]

What does it looks like?

To hide a page or section, set the hidden property to true in the front matter:

title: My Page
hidden: true

What is the hidden property used for?

When the hidden property is set to true, the following behaviors will take effect:

• Pages or sections with this property will still be rendered as usual.
• They will not be indexed in search index.
• They will not be included in the sitemap.xml.
• They will not be included in atom/feed.
• Template authors can check for this property and include noindex in the page metadata tag or exclude it from the menu/navigation.
• If the property is set for a section, the section itself and entirely nested content also take effect.

The purpose is to hide the pages or sections from search engine crawlers while still allowing access via the exact URL.

Use Case

This feature is useful in scenarios such as:

• Keeping certain pages hidden from normal visitors, only accessible with a shared URL.
• Excluding pages protected by authentication from appearing in the sitemap.
  For example, the Home page can be viewed and indexed, while private content remains hidden and excluded from the sitemap.

[mwcz]

draft already covers most of these needs, but I'm not sure if it can be used on sections.

[tuyen-at-work]

@mwcz draft is use for different purpose. For example on local you create a markdown file but decide that it is not ready for delivery then you mark it as draft.

If you deliver the site with zola build --drafts then the draft pages will be rendered, but they also appear in sitemap/feed/atom/search index. Therefore it does not serve the same purpose.

[Keats]

I'm not sure we want that, that's pretty niche. You can accomplish the same by writing your own sitemap template and checking for a specific arg in extra to not include them and when rendering section (breaks down though if you have hidden things in a paginated section).

[mwcz]

If you deliver the site with zola build --drafts then the draft pages will be rendered, but they also appear in sitemap/feed/atom/search index. Therefore it does not serve the same purpose.

zola build --drafts
zola build

Then deploy. I used to do that with Hugo and it works just as well with Zola. The only thing it doesn't cover is hidden sections.

[RomainMazB]

@Keats additionally to the sitemap, the hidden property feature would handle the fact that the internal search doesn't index these pages.

ATM for the bevyengine/bevy docs, we do not use the internal search feature so we can go with the solution you mentioned using extra and modifying the sitemap.xml template (cc @tuyen-at-work):

[extra]
status = 'hidden'

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    {% for sitemap_entry in entries %}
    {% if not sitemap_entry.extra.status %}
    <url>
        <loc>{{ sitemap_entry.permalink | escape_xml | safe }}</loc>
        {% if sitemap_entry.updated %}
        <lastmod>{{ sitemap_entry.updated }}</lastmod>
        {% endif %}
    </url>
    {% endif %}
    {% endfor %}
</urlset>

But if we integrate the search engine, we will need this feature. With the search engine problematic in mind, would you accept a PR to tackle this?

[glhewett]

I came across this feature request, and this is exactly what I was hoping for. I am 48 hours into using zola, and I am really enjoying it. I have some experience with some other static site generators, so I am not a completely new to this.

I really like the idea of the hidden flag. I think I have seen it in other generators that I have used. I was expecting the in_search_index to behave like what is being propose here. I was sad to see that it was not that.

My use case is simple. I like to do my documentation in markdown -- as most of us that would be using this tool. I would like to create documentation, host it on my site, and choose who can have a link to it. It is kinda like a gist that is hosted on my domain.

@Keats -- I saw that you think this is rather niche, and I agree that many might not be using it. However, I want to provide my attempt to warm you up to accepting a PR if I provide one.

The following is an example of how I might want to use this. Notice that I am obfuscating the URL to make it harder to guess.

+++
title = "First Private Page"
slug = "m3SrhoqK"
date = 2024-02-04
hidden = true
+++

# Private Page

This is my first private page.

[Keats]

Looks like it's not that niche!

How does it work for sections? Hidden sections implies all hidden pages with no way to un hide them?

[tuyen-at-work]

Hey @Keats,

I'd like to highlight that there's currently support for route-based protection in some static hostings. For instance, Azure Static Web Apps offers the ability to secure routes with roles. I believe platforms like Netlify and Vercel may also have similar features.

It would be great if content within protected routes doesn't get exposed in public links, such as sitemaps, search indexes, or atom feeds.

This functionality also allows us to seamlessly include both public and private content on the same website without worrying about unauthorized access.

[Keats]

Well you would have to use that new hidden field and map that with your host if needed

[tuyen-at-work]

Yes I understand Zola shouldn't worry about config the cloud hosting, just want to highligh that some clouds support route-based protection. I think the use-case is popular enough that Microsoft decided to supports it for a long time already.

This feature, if being implemented, will enable us to utilize that feature nicely, i.e. if I decide to protect a route in the host, all I need to do from zola side is set the route to hidden and it will be out of the world's knowledge.

[darrenmeehan]

There's some content I'd use for this capability for if it existed!

I think if this feature is added it should be documented clearly that these pages are still publicly accessible.

At risk of feature creep, would it be useful to output a count of hidden pages?

[pietroalbini]

Looks like it's not that niche!

Is this something you'd accept a pull request for?

How does it work for sections? Hidden sections implies all hidden pages with no way to un hide them?

That could work, otherwise we could have three states for hidden: missing/true/false, where missing inherits from the parent section.

[Keats]

Is this something you'd accept a pull request for?

Yes, as long as there's a clear solution for section with hidden=true.

That could work, otherwise we could have three states for hidden: missing/true/false, where missing inherits from the parent section.

So something like

• if section hidden=true
• then a page/section inside can have hidden=false to be displayed?

[glhewett]

I am working through creating a PR. I am not going very fast, so if someone beats me to it, I think that is great. Two thoughts that I had while I am working on this:

1. I was thinking that an implementation in sections might be confusing. For a blog with a hidden post, I would just set the page to hidden and allow the section to be displayed. The section should obey the pages' hidden=true flag and not render the link. For a section that contains only private content, I would like to just set the render=false meta in the section. This means that every hidden page would need to be explicitly set. Does this sound reasonable?
2. I was thinking that the filter logic that is implemented into the templates may cause a user error or a regression. I am looking to make sure that the page filter for hidden=true is implemented in the code that calls the template.

@pietroalbini, I am happy to share what I have if you want to make this go faster.

I have done sitemaps and feeds. I now need add the filter to the section, search and write some tests.

[Keats]

For 1 I would expect to be able to set hidden=true in the section but have pages or subsections in them with hidden=false and let them render? Not sure how that works in practice though.

[clarfonthey]

I think that defaulting the hidden = true status of a section but still allowing you to override it works. Seems pretty consistent to me.

[tuyen-at-work]

Hi @Keats @glhewett @pietroalbini,

My initial request did not cover the functionality to override the hidden property for nested pages (meaning if a section is hidden, all child pages are also hidden). However, including this feature would be a valuable addition.

If you are currently working on supporting the override of properties on child pages, please also consider implementing it for child sections.

[izissise]

So I struggled with this today https://github.com/getzola/zola/issues/742#issuecomment-1961164399

But I found a workaround with dates:

    {% set ts_now = now()|date(format="%s")|int %}
    {%- for page in pages %}
    {% set ts_page = page.date|default(value=0)|date(format="%s")|int %}
    {% if ts_page < ts_now %}
    html...
    {% endif %}
   {%- endfor %}

The logic can be apply on sitemap atom etc... This way you can just set date = "3024-02-23" on a post that you don't want published.

[alexpetros]

I have this exact need from Zola, and specifically, I want it so that I can publish drafts at unlisted URL. That way I can send them to friends to review, but not have them land in my RSS feed.

[alexpetros]

So I struggled with this today #742 (comment)

But I found a workaround with dates:

    {% set ts_now = now()|date(format="%s")|int %}
    {%- for page in pages %}
    {% set ts_page = page.date|default(value=0)|date(format="%s")|int %}
    {% if ts_page < ts_now %}
    html...
    {% endif %}
   {%- endfor %}

The logic can be apply on sitemap atom etc... This way you can just set date = "3024-02-23" on a post that you don't want published.

Until this feature exists, I was able to improve upon this a little bit by using the [extra] field of the front matter. So the full steps are, from scratch:

1. Copy the atom.xml file from the zola source into your own templates/ directory.
2. Add {%- if not page.extra or page.extra.hidden != true %} below {%- for page in pages %} in the atom.xml file
3. Add {%- endif %} above the appropriate {%- endfor %}
4. Repeat steps (2) and (3) in whichever templates you render the list of blog posts
5. Add the following to the front matter of the pages your want hidden:

[extra]
hidden = true

And you'll have a page that renders at the correct link, but does not show in your RSS feed or list of posts. Great for sending people unlisted drafts. If/when the feature gets added (I'll try to PR one if I find the time, but I recognize everyone's busy), you can just move that hidden = true out of the [extra] section.