Discussion: Add support for category-level data overrides

[james-nash]

As a design system creator using Pattern Lab, I want to be able to override data values at the category level, so that I don't have to manually copy-paste (and maintain) overridden values that are shared by all patterns in that category

---

@tburny originally enquired on the #patternlab-node Gitter if there was a way to override data.json for a single directory only. The motivation was for projects where different brands share the same UI components and layout but differ only in the skinning of the UI.

I believe this feature would also be useful for the pages category, since replacing data values to swap lorem ipsum text and wireframe images for realistic content often needs to be repeated per page (since different pages may be re-using the same data values). Currently the only means to do this is to create patternName.json files for each pattern under the pages folder and copy-paste the relevant values into each. When you throw listitems overrides into the mix this gets tedious very quickly. Maintenance is also a pain since you need to update multiple copies of the same value.

Being able to have a single file that applies to an entire directory would therefore be a big convenience win in these scenarios.

Perhaps something like this:

|
+- 04-pages/
|     |
|     +- 99-foo.mustache  # Usual stuff...
|     +- 99-foo.json
|     +- 99-foo.listitems.json
|     +- 99-foo.md
|
+- 04-pages.md
+- 04-pages.json  # NEW data that can override root data.json for every pattern inside 00-atoms/
+- 04-pages.listitems.json # NEW listitems equivalent of the above

The expectation would be that to find the data values used when rendering 99-foo.mustache PatternLab would first look in 99-foo.json (as it already does), then 04-pages.json (this is the new feature) and finally data.json.

The same technique should also work for listitems. I.e. 99-foo.listitems.json overrides 04-pages.listitems.json, which in turn overrides listitems.json.

Note: Some discussion occurred in patternlab-node issue #599 before it was moved here.

/cc @pattern-lab/voting-members

[tburny]

Keeping the data under _data would prevents potential name clashes with existing templates. Actually its sort of "global" data on a directory level.

[bradfrost]

@james-nash Thanks for this suggestion. Just so I can wrap my head around this, you're trying to accomplish theming that can (maybe even radically) adjust the skin of the UI? How I've done this in the past is to create pseudo-patterns for theme-specific pages and components. So for instance, I might have a file called dashboard.json in my pages, and then would also create a pseudo-pattern called dashboard~theme-blue.json, whose contents might look like this:

{
    "bodyClass": "theme-blue"
} 

This would add the theme class to to the body and apply the theme styles. Additionally, I can accomplish this at a component level. For instance, I may have my button component, but then might also create a blue theme button, like so: button~theme-blue. I can either then use the same method as before (adjusting the bodyClass to apply the theme styles) or to create a modifier to the button component itself, like so:

{
    "styleModifier": "btn--theme-blue"
} 

I'm less privy to the technical workings of data inheritance and all that, but I just thought I'd chime in with how I accomplish theming in my own work.

[bmuenzenmeyer]

Thanks for your thoughts @bradfrost - it's always appreciated to see how you've tackled use cases in PL.

If I understand @james-nash correctly, however, we would have, say, some navigation component:

molescules-nav

<nav>
{{#links}}
 <a href="{{url}}">{{text}}</a>
{{/links
</nav>

which is included in:

pages-home

<h1>{{siteName}}</h1>
{{molecules-nav}}
...

with an accompanying json file

pages-home.json

{
 "links": [
   { "url": "foo", "text": "bar"},
    ...
 ],
 "homePageThing": "baz"
}

Of course something like siteName can be defined globally inside source/_data/data.json, but because data inheritence does not work (by design) one cannot define a

molecules-nav.json`

{
 "links": [
   { "url": "foo", "text": "bar"},
    ...
 ]
}

and expect its data to carry over to the pages data.

Users wanting other-page to share the same navigation links as pages-home would be forced to repeat the links key inside other-page.json or (worse) promote it to source/_data/data.json (which then puts real data, or even pattern links in your molecules-nav patterns and templates too). I suppose this goes against strict atomic design at that point, because molecule/organism/template data has page-level data inserted into it.

---

So, I understand the merits of the use case. I don't think strict data-inheritence as people are often requesting resolves it either (above reason). Directory-level json may in fact have a place, then.

---

I don't think this directory json belongs inside source/_data because json inside that folder is allowed to be broken up and concatenated into one large global dataset, and I don't want us to have to special case against pattern group directories.

I like the idea of putting this data alongside the directory, similar to pattern group markdown.

source/patterns/
  foo/
  foo.md //optional pattern group markdown
  bar/
  pages/
  pages.json // optional pattern group data

The hierarchy of data would then be something like:

patternParameter(s) >>pattern.json>>patternGroup.json>>source/_data/data.json`

Thoughts?

[bmuenzenmeyer]

(Essentially, there is contention between templates and pages, because one craves sample data, and the other real data, but they both use the same patterns.)

[james-nash]

@bradfrost Thanks for your comments. As @bmuenzenmeyer says, it's always useful to hear how others tackle things. I think the use-case you outlined is what prompted @tburny to request this feature in the first place.

FWIW, my own use-case is slightly different. I think Brian's already covered it, but just for the avoidance of doubt:

Let's say I've got a root data.json, with some generic text, images, etc like this:

{
    title: "Titeley goesey herey",
    description: "Lorem ipsum sit dolar...",
    navLinks: [
         { "label": "Nav link 1", "url": "#" },
         { ... },
         ...
    ],
    img: {
        "16x9": { "src": "../../images/16x9-placeholder.png", "alt": "Placeholder for 16x9 aspect ratio image" },
        "1x1": { ... },
        ...
    },
    ...
}

I'd use that throughout my atoms, molecules, organisms and templates. However, for my pages, I'd like to substitute realistic data. On my projects, I've ended up with many pages and a lot of the data used for them is identical.

For instance, let's say the navLinks data is used by a navigation component in the page header. On my pages I might want to replace "Nav link 1", "Nav link 2" etc. with something realistic like "Home", "About" etc. Since the header gets used on all pages, I want to override those same values for every page I have.

Right now, I need to repeat that individually for every page I have. However, if I could do this once for the whole pages folder (as per this proposal), it would save me a lot of copy-pasting and also make future updates/corrections that much easier. I could do something like this once to replace the data used across all my pages:

_patterns/04-pages.json:

{
    navLinks: [
         { "label": "Home", "url": "link.pages-home" },
         { "label": "About", "url": "link.pages-about" },
         { ... },
         ...
    ],
    img: {
        "16x9": { "src": "../../images/examples/letmebeyourguide.jpg", "alt": "Cat wearing a bandana riding a fire-breathing unicorn" },
        "1x1": { ... },
        ...
    },
    ...
}

This pain (and potential gain) is even more pronounced when working with listitems.json, since you're creating/updating 12x the amount of data.

For example: A recent project I worked on was for a news site, so most pages featured various lists of links to articles. We were using listitems to provide individual images, titles and blurbs for those link lists. When inserting realistic values for our pages, we had to duplicate those many times over.

[tburny]

Actually this looks a bit similar to #28.

To wrap things up, we have to distinct use cases:

• Overriding data.json mock data with real contents in pages
• Theming a website (same patternlab, multiple layout variants and real contents)

Note

Actually it might be good to have this feature for sub-types, too. For instance if I have 00-atoms/00-buttons and a variety of button styles (lets say Submit, Cancel, various sized, red, blue, you name it) with the same dummy text, I could set a locally scoped text value for all buttons at once. Still I would not want that "text" in the 00-atoms as maybe I have some text inputs where I want to use a different value. This would help users to obey the DRY principle AND improve performance due to the reduced amount of data that has to be parsed from disk (see https://github.com/pattern-lab/patternlab-node/issues/316 and https://github.com/pattern-lab/patternlab-node/issues/604).

The real benefit comes in with my original request and what @james-nash said: Currently we have 05-pages and 06-szt as a category for theming http://muenchenticket.de and http://sueddeutsche-tickets.de although 05-pages/00-default and 05-pages/01-szt show a much nicer structure. An alternative might be a data-pseudopattern (05-pages.json and `05-pages~szt.json), but it could be programatically hard to know which one to use when and it would be more like a pattern group.

[bmuenzenmeyer]

@bradfrost or @dmolsen do you have any additional input?

[bradfrost]

I'm still a bit confused as to the need for category-specific data, but if it makes sense by all means don't let me hold everyone up.

The above examples appear to be able to be achieved by creating global data values and pattern-specific values, which can even be chunked out into separate files. For instance, our _data/ folder for a recent project looked like this:

- buttons.json
- colors.json
- data.json
- forms.json
- icons.json
- images.json
- lists.json
- navigation.json
- tables.json

We defined the default data and then overwrote that data at the pattern and page levels.

For instance, let's say the navLinks data is used by a navigation component in the page header. On my pages I might want to replace "Nav link 1", "Nav link 2" etc. with something realistic like "Home", "About" etc. Since the header gets used on all pages, I want to override those same values for every page I have.

For this instance, I'd say that "Home", "About", etc should be the default data if it's being used so many times and is applied to all pages. You can then override that default data at the pattern level. For instance, in header.json (living next to header.mustache or whatever language you're using), you could define:

"navLinks" : [
   { "label" : "Nav link 1" },
   { "label" : "Nav link 2" }
]

Essentially "Nav link 1" and "Nav link 2" are the overrides and not the default. Does that make sense?

Again, it might still make sense to have category-level overrides as another tool in the toolkit, but I just wanted to point out that these scenarios seem to be solvable using existing functionality. Thanks!

[bmuenzenmeyer]

@bradfrost maybe this one final piece of info will help. You could call it an example of this "problem" in the wild.

If you look at the data.json we define for the demo: https://github.com/pattern-lab/starterkit-mustache-demo/blob/master/dist/_data/data.json#L54-L89

you'll notice that these links are the "production links" as in the finished pages.

This means that people constructing or view the navigation patterns (which are supposed to be real-content-agnostic® according to atomic design) will be seeing the real-content at the nav level and the template level, not placeholder data. (see http://demo.patternlab.io/?p=templates-homepage page and http://demo.patternlab.io/?p=organisms-header patterns)

This is in opposition of a more organic workflow, where one would likely define nav items as junk like you defined above...

"navLinks" : [
   { "label" : "Nav link 1" },
   { "label" : "Nav link 2" }
]

only to override with page-level data as it gets built out. Doing this, however, creates the problem this spec enhancement is attempting to solve, by not having to repeat the navLinks override for each page we would create.

One solution to this problem is defining the global nav inside data.json as you and the demo suggest, but then we are back where we started.

---

It's good to hear that you are open to the idea of having another tool. If this is brought to a vote, your vote is required unless Dave returns to a more active state.

[dmolsen]

+1. Seems like an obvious change.

On Feb 8, 2017, at 1:18 PM, Brian Muenzenmeyer notifications@github.com wrote:

@bradfrost maybe this one final piece of info will help.

If you look at the data.json we define for the demo: https://github.com/pattern-lab/starterkit-mustache-demo/blob/master/dist/_data/data.json#L54-L89

you'll notice that these links are the "production links" as in the finished pages.

This means that people constructing or view the navigation patterns (which are supposed to be real-content-agnostic® according to atomic design) will be seeing the real-content at the nav level and the template level, not placeholder data.

This is in opposition of a more organic workflow, where one would likely define nav items as junk like you defined above...

"navLinks" : [ { "label" : "Nav link 1" }, { "label" : "Nav link 2" } ] only to override with page-level data as it gets built out. Doing this, however, creates the problem this spec enhancement is attempting to solve, by not having to repeat the navLinks override for each page we would create.

One solution to this problem is defining the global nav inside data.json as you and the demo suggest, but then we are back where we started.

It's good to hear that you are open to the idea of having another tool. If this is brought to a vote, your vote is required unless Dave returns to a more active state.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[bmuenzenmeyer]

Cool. I will write up a final update and then prompt for a vote.

[bradfrost]

+1

http://giphy.com/gifs/television-the-simpsons-election-2012-LnlvMLQPwEdsA

[bmuenzenmeyer]

Brad and Dave have already weighed in, but here's the more formal writeup.

Use case

A user wants to include data on the pattern group level which will be applied to all patterns of that pattern group, so as to provide unique and DRY data

---

System Input

Consider the following files underneath source/_patterns/

|
+- 04-pages/
|     |
|     +- foo.mustache  # Usual stuff...
|     +- foo.json
|     +- foo.listitems.json
|     +- foo.md
|
+- 04-pages.md
+- 04-pages.json  # NEW data that can override root data.json for every pattern inside 04-pages/
+- 04-pages.listitems.json # NEW listitems equivalent of the above

Pattern Lab processing would recognize and load these new files into the library for later output. .yaml could also be supported should that platform implement yaml for data. Note, this pattern group level .json/.yaml is housed next to the directory which it applies to.

One pattern group's data does not get inherited into other pattern groups' data.

System Output

When processing any individual pattern, the hierarchy of data-inheritance is as follows, before being fed in for rendering:

patternParameter(s) >> pattern.json >> patternGroup.json >> source/_data/data.json

Extra

The timeline for this feature is determined by the availability of platform maintainers.

Tagging core and spec-enhancement because it'll be implemented in core and does not exist today.

This vote will close at 12PM on February 23 or once two voting members have voted yay or nay.

/cc @pattern-lab/voting-members

[bmuenzenmeyer]

Vote: 👍

[bradfrost]

👍

[bmuenzenmeyer]

Vote passes. Thanks for the input @james-nash and @tburny