Feature request: Using page factories for large-scale tree population

[ababic]

For a lot of projects, it can be helpful to establish the general 'page structure' of a site early on.

In the past, I've used fixtures for this, but I find them cumbersome. Every time you decide to switch up your model classes or fields, the fixtures remain unaware and require either a painful 'manual update' process, or a painful 'manually populate the tree locally and re-dump everything' process.

In a recent project, I decided that, since we were mostly defining a factory for every page type, we could use factories instead! I came up with the following:

from typing import Optional, Sequence

from django.template.defaultfilters import slugify
from wagtail.core.models import Page

from my_app.standardpages.factories import GeneralPageFactory

class CreatePageNode:
    """
    Used to represent a 'Wagtail page' that must be created.
    """

    default_factory_class = GeneralPageFactory

    def __init__(
        self,
        title: str,
        factory_class: type = None,
        *,
        children: Optional[Sequence["CreatePageNode"]] = None,
        **kwargs,
    ):
        self.title = title
        self.create_kwargs = kwargs
        self.factory_class = factory_class or self.default_factory_class
        self.children = children or []

    def create(
        self, parent: Page = None, create_subpages: bool = True, **kwargs
    ) -> None:
        create_kwargs = {**self.create_kwargs, **kwargs}
        create_kwargs.setdefault("title", self.title)
        if parent is None:
            parent = create_kwargs.get("parent") or Page.objects.first()

        try:
            page = (
                parent.get_children()
                .get(slug=create_kwargs.get("slug", slugify(create_kwargs["title"])))
                .specific
            )
            created = False
        except Page.DoesNotExist:
            create_kwargs.setdefault("parent", parent)
            page = self.factory_class.create(**create_kwargs)
            created = True

        if create_subpages:
            for child in self.children:
                child.create(parent=page, create_subpages=True)

        return page, created

This can then be used to define a 'page tree to create', like so:

from .utils import CreatePageNode as p

PAGES_TO_CREATE = p(
    "Home",
    HomePageFactory,
    slug="home",
    children=(
        p(
            "About us",
            children=(
                p("Accessibility Statement", specific_field_value="foo"),
                p("Collection", factory_property_value="bar"),
                p("Contact Us", slug="contact"),
                p("Corporate Support"),
                p("Frequently Asked Questions", slug="faqs"),
                p("Governance"),
                p(
                    "Policies and Procedures",
                    slug="policies-and-procedures",
                    children=(
                        p("Privacy Policy"),
                        p("Terms and Conditions"),
                    ),
                ),
                p("Projects", ProjectIndexPageFactory),
                p("Working for us"),
            ),
        ),
        p(
            "Support us",
            slug="support-us",
            children=(
                p("Become a member"),
                p("Become a patron"),
                p("Donate"),
            ),
        ),
        p(
            "Press",
            PressReleaseIndexPageFactory
        ),
        p(
            "Whats On?",
            EventIndexPageFactory,
            children=(
                p("Workshops", EventCategoryPageFactory, children=(
                    p("Example workshop", EventPageFactory, event_style="workshop"),
                ), 
                p("Food & drink", EventCategoryPageFactory, children=(
                    p("Example cocktail evening", EventPageFactory, event_style="food"),
                ), 
                p("Music", EventCategoryPageFactory, children=(
                    p("Jazz club", EventPageFactory, event_style="music", music_style="jazz"),
                    p("Rock city", EventPageFactory, event_style="music", music_style="rock"),
                ),
            )
        ),
    ),
)

This can easily be incorporated into a management command that can trigger creation of all of these pages, like so:

from django.core.management.base import BaseCommand

class Command(BaseCommand):
    def handle(self, *args, **options)
        PAGES_TO_CREATE.create(create_subpages=True)

The process safely avoids creating/overwriting pages that already exist - meaning it can be rerun at a later time to add new sections (as the page types are developed)

[ababic]

The implementation above could easily be built upon to:

• Allow for 'lazy' factory specification (using a string, and only importing/loading the factory class when create() is called)
• Use settings to control the default_factory_class and data source for the management command (specified via dotted python path strings)

[bcdickinson]

Intriguing idea, but I'm not clear what would need to be built into wagtail-factories to support this use case. At first glance, a lot of your example looks like it'd be very specific to a particular project and there isn't a lot of code around just invoking the project's own factory classes.

Could you be more specific or maybe raise a PR?

[easherma-truth]

I think part of the issue is the sparse documentation around calling the factories (especially streamfield ones) in ways that will work. Like linking to something like body__0__carousel__items__0__label='Slide 1', in tests doesn't make a ton of sense by itself.

[ababic]

@bcdickinson thanks for getting back to me. The idea here is that the CreatePageNode is generic enough to be used to create any number of pages, in any structure, and of any type. So perhaps that could find a home within the project as a util of sorts, with the other code examples possibly being translated into documentation on how to use it?

Or maybe the entire thing would go into the docs, as hinted at by @easherma-truth?

Or, if you don't see a place for this sort of thing in the project at all, that's equally fine. I just thought it would be helpful for others.