Pagination can't handle nested collections

[MattMcAdams]

Describe the bug

If you create a nested collection using eleventyConfig.addCollection, and try to use that in pagination, it will not be able to complete the build and throw an error.

To Reproduce

Create a nested collection. Here is a smaller bit of what I was playing with:

function getTagList(collection) {
  let tagSet = new Set();
  collection.forEach((item) => {
    (item.data.tags || []).forEach((tag) => tagSet.add(tag));
  });
  return filterTagList([...tagSet]);
}

eleventyConfig.addCollection("projects", function (collectionAPI) {
  let PROJECTS = collectionAPI.getFilteredByGlob("./src/projects/*.md");
  let collection = {};
  collection.all = PROJECTS;
  collection.tags = getTagList(PROJECTS);
  return collection;
});

I explain more about what I was doing in my blog post.

And try to use it in pagination front-matter:

---
layout: layouts/no-sidebar.njk
pagination:
  data: collections.projects.tags
  size: 1
  alias: tag
eleventyComputed:
  title: Tagged “{{ tag }}”
permalink: /projects/tags/{{ tag | slug }}/
---

You'll get:

Error: Could not find pagination data, went looking for: collections.projects.tags

Expected behavior

I'd expect it to work the same way it does when I create a collection like this:

// Create a collection for post tags (required for the generation of tag pages)
eleventyConfig.addCollection("projectTags", function (collectionAPI) {
  let PROJECTS = collectionAPI.getFilteredByGlob("./src/projects/*.md")
  return getTagList(PROJECTS);
});

For example, I can use:

{% for tag in collections.projects.tags %}
{% set tagUrl %}/projects/tags/{{ tag | slug }}/{% endset %}
   <li><a href="{{ tagUrl | url }}">#{{ tag }}</a></li>
{% endfor %}

And I get a list of all the tags, the same as if I did collections.projectTags

Environment:

• OS and Version: Mac
• Eleventy Version: 1.0.0

Additional context

I fully understand there's likely no reason to create collections like this, but I was just playing with the collections system and thought this was unexpected behavior. This may be related to issue #1349, I wasn't completely sure if it was the exact same problem though, so I opened a new issue for it.

[kuwts]

Running into the same issue, did you end up solving this somehow?

[pdehaan]

I've never been able to get this to work. I think .addCollection() expects an array.

I think your two possible workarounds are

1. Create two collections; one for pages (ie: collections.projects), and one for tags (ie: collections.projects:tags).
2. Use a single collections.projects collection and modify the pagination data using the before() callback (see below).

---
# src/index.liquid
# This file has no front matter, we specify it below in `src/index.11tydata.js` since we want
# JavaScript style front matter.
---

<h1>{{ title }}</h1>

// src/index.11tydata.js
const { getTagList } = require("../lib");

module.exports = {
  pagination: {
    // Loop over our `collections.projects` collection [of pages]...
    data: "collections.projects",
    size: 1,
    alias: "tag",
    before(data) {
      // ... but first, convert the collection of pages to a tag list.
      return getTagList(data);
    }
  },
  eleventyComputed: {
    title(data) {
      return `Tagged “{{ tag }}”`;
    },
    permalink(data) {
      return `/projects/tags/${ this.slugify(data.tag) }/`;
    },
  }
};

// .eleventy.js
module.exports = function (eleventyConfig) {
  eleventyConfig.addCollection("projects", function (collectionAPI) {
    return collectionAPI.getFilteredByGlob("./src/projects/*.md");
  });

  // This custom filter isn't currently used anywhere (since I couldn't figure out how to get access to
  // global filters in the `before()` callback), but it might be useful. 🤷 
  eleventyConfig.addFilter("getTagList", require("./lib").getTagList);

  return {
    dir: {
      input: "src",
      output: "www",
    }
  };
};

// lib.js
module.exports.getTagList = getTagList;

function getTagList(collection = []) {
  let tagSet = new Set();
  collection.forEach((item) => {
    (item.data.tags || []).forEach((tag) => tagSet.add(tag));
  });
  // Convert the Set to a vanilla Array.
  return [...tagSet].sort();
}

So I don't think it's as pretty as the original concept of nested collection objects, but it seemed to work. Another area to explore would be seeing if you could paginate over eleventyComputed data. If so, it might save some awkwardness around the before() callback and access to filters/data.

[pdehaan]

Per https://github.com/11ty/eleventy/issues/1110, and my limited testing, I don't think you can paginate over eleventyComputed values.

---
## This doesn't work and won't write out any pages (in v1.0.0)
pagination:
  data: computer
  size: 1
  alias: ok

computer: []

eleventyComputed:
  computer:
    - fizz
    - buzz
---

>> {{ ok }}

<pre>{{ computer }}</pre>

[dwighthouse]

This is almost exactly my use case. I generate a complex collection to do manual pagination of all posts with every tag (a pagination of paginations), and I also generate a collection of all tags for my "Tags" page. The calculations for each of these collections is very similar, sharing much logic. I could be doing this in a single loop, rather than two. Furthermore, I have some non-trivial computation attached to tags. I'm generating tag associations and implications, augmenting whatever tags are in the front matter with a more complete set with a hierarchy of associations. There's no point in doing this twice for every time collections are generated.

If the pagination system could handle the data field as if it was a normal JS object dereference, rather than what I assume is "whatever is after the dot, as a string" to find the collection, it would get the appropriate data. Like this:

---
pagination:
    data: collections.unified_tags.paginated_tag_pages
    size: 1
    alias: tag
---

I attempted the before solution, but it didn't generate any output.

[pdehaan]

I attempted the before solution, but it didn't generate any output.

I think I got it w/ before() but it was a bit confusing&hellip (like getting an empty collections object if I was looping over a local array versus a collection);

First of all, I manually created a custom collection in .eleventy.js config file w/ a nested object:

  eleventyConfig.addCollection("nested_collection", function (collectionApi) {
    return {
      "prop1": [{name:"c.one"}, {name: "c.two"}, {name: "c.three"}],
      "prop2": [{name: "c.six"}, {name: "c.seven"}, {name: "c.eight"}],
    }
  });

Now I have a /src/index.njk file and /src/index.11tydata.js data file (since I prefer that to ---js style frontmatter):

// src/index.11tydata.js
const _get = require("lodash.get");

module.exports = () => {
  return {
    key: "prop2",
    pagination: {
      data: "collections.nested_collection",
      size: 1,
      alias: "p",
      before(data=[], cfg={}) {
        // `data` is the array of keys in the `pagination.data` object.
        if (!data.includes(cfg.key)) {
          throw new Error(`Unknown key: "${cfg.key}"`);
        }
        const key = [cfg.pagination.data, cfg.key].join(".");
        return _get(cfg, key);
      }
    },
    permalink: "string/{{ p.name }}/",
  }
};

---
# src/index.njk
---
<pre>{{ p.name }}</pre>

When you paginate over collections.nested_collection, you'll get back an array of keys or values (default is resolve: keys). By default the data argument in the before() function will be an array w/ the following values ['prop1', 'prop2']. So I can specify the specific key I want in the front matter using key: "prop2" and then use lodash.get to convert that into collections.nested_collection.prop2. And I also added some really cheap validation code to make sure my key is a valid property in the collections.nested_collection collection.

[dwighthouse]

@pdehaan I'm not sure why this worked when my solution didn't. I made sure to send a valid array as the return value for my before() function, but it acted as if it was an empty list (which the "nested_collection" would have been, I suspect). Is there something special about having the key property? Couldn't you just return a dynamically generated list from before() with no special call outs to anything?

[pdehaan]

🤷

If I was looping over a vanilla array from my front matter instead of a "collections.whatever" I did notice that the before() function was returning an empty object for collections, so something definitely felt a bit funny there.

---
myarray:
  - { name: "one" }
pagination:
  data: "myarray"
  size: 1
  alias: p
---

So that would work, but then got weird if I added in a before(data, cfg) hook and tried accessing cfg.collections.

Nothing special about that arbitrary key front matter object. I even tried moving it into pagination.key to be closer to pagination.data and it worked, but felt a bit awkward since I was then polluting the pagination object w/ custom keys.

Yeah, you can return a random array from before() if you don't want to try and use the built-in collections, etc. This should work, as long as collections.nested_collection has at least one item. If it's an empty array, it seems to skip the before() function.

module.exports = () => {
  return {
    pagination: {
      data: "collections.nested_collection",
      size: 1,
      alias: "p",
      before(data=[], cfg={}) {
        return [{name: "sEVEN"}, {name: "eIGHT"}, {name: "nINE"}];
      }
    },
    permalink: "string/{{ p.name }}/",
  }
};

But I'd have to try it with a more real-world case (like returning an object w/ a collection of templates and then collection specific tags) and see how it works. Generally I'd just create two separate collections like collections.sports and collections.sports_tags or something to avoid needing workarounds (at the expense of an extra collection).

[pdehaan]

Pushed a repo with a clearer [and hopefully more real-world] example: https://github.com/pdehaan/11ty-object-pagination

[dwighthouse]

Thanks @pdehaan. I'll have a look at this in detail when I get the chance.

[hidegh]

@dwighthouse i was working on such nested pagination and made it work, fairly general, although since 0.11 version there might be better solutions.

check out this https://github.com/hidegh/jamstack-eleventy-custom, there's also a live sample hosted, where you can check how the categories use this nested approach. the trick was to create a generic structure so i can have a reusable navigation.