SPIKE: Explore wizard-like custom form flow in Drupal

[ryguyk]

Overview

The default OOB node-edit/node-create form in Drupal is not the ideal UX for our product. We need to explore the feasibility of creating a more custom solution.

Acceptance Criteria

• [x] Determine the feasibility of a wizard-like UX
• [x] Identify possible risk
• [x] Demonstrate discovery "prototype"
• [x] Provide a decision or recommendation with regard to whether this should be done

[ryguyk]

One question that I've been exploring is what happens - in the default flow - with revisions, especially with regard to paragraphs. More specifically, I wanted to know if a new paragraph revision was created on node save even if the paragraph information had not changed (or, rather, would the new node revision point to the existing paragraph revision?). The answer to that question appears to be that a new revision is created regardless of whether data has been changed, and that goes for both the node and the paragraphs. Here is a query I used to determine that:

SELECT 
    n.nid, 
    n.type,
    nr.vid,
    nr.revision_log,
    fc.field_chapters_target_id, 
    fc.field_chapters_target_revision_id, 
    pi.type,
    pi.id,
    pir.revision_id,
    pft.*,
    prft.*
FROM node n
LEFT JOIN node_revision nr ON n.nid = nr.nid
LEFT JOIN node_revision__field_chapters fc ON nr.vid = fc.revision_id
LEFT JOIN paragraphs_item pi ON fc.field_chapters_target_id = pi.id
LEFT JOIN paragraphs_item_revision pir ON fc.field_chapters_target_revision_id = pir.revision_id
LEFT JOIN paragraph__field_title pft ON pft.entity_id = pi.id
LEFT JOIN paragraph_revision__field_title prft ON prft.revision_id = pir.revision_id
WHERE n.type = 'digital_form'
ORDER BY nid DESC, nr.revision_timestamp DESC;

Looking at a notable piece of the result set, we can see that, despite not changing anything, we get a new node revision and a new paragraph revision referenced by field_chapters.

{
    "rows":
    [
        {
            "nid": 70917,
            "type": "digital_form",
            "vid": 906643,
            "revision_log": "2 - Didn't change anything",
            "field_chapters_target_id": 157899,
            "field_chapters_target_revision_id": 1415559,
            "type": "digital_form_name_and_date_of_bi",
            "id": 157899,
            "revision_id": 1415559,
            "bundle": "digital_form_name_and_date_of_bi",
            "deleted": 0,
            "entity_id": 157899,
            "revision_id": 1415559,
            "langcode": "en",
            "delta": 0,
            "field_title_value": "Name and Date of Birth 2",
            "bundle": "digital_form_name_and_date_of_bi",
            "deleted": 0,
            "entity_id": 157899,
            "revision_id": 1415559,
            "langcode": "en",
            "delta": 0,
            "field_title_value": "Name and Date of Birth 2"
        },
        {
            "nid": 70917,
            "type": "digital_form",
            "vid": 906642,
            "revision_log": "1 - Initial save",
            "field_chapters_target_id": 157899,
            "field_chapters_target_revision_id": 1415558,
            "type": "digital_form_name_and_date_of_bi",
            "id": 157899,
            "revision_id": 1415558,
            "bundle": "digital_form_name_and_date_of_bi",
            "deleted": 0,
            "entity_id": 157899,
            "revision_id": 1415559,
            "langcode": "en",
            "delta": 0,
            "field_title_value": "Name and Date of Birth 2",
            "bundle": "digital_form_name_and_date_of_bi",
            "deleted": 0,
            "entity_id": 157899,
            "revision_id": 1415558,
            "langcode": "en",
            "delta": 0,
            "field_title_value": "Name and Date of Birth 2"
        },
    ]
}

This knowledge will help us plan how we might handle save-in-progress as editors progress through a wizard-like form.

[ryguyk]

Findings

Overview

This spike - at its core - attempted to determine the feasibility of "ejecting" from the default node-edit/node-create forms in favor of a custom step-by-step/wizard experience. In short, the spike discovered that this sort of experience seems very doable.

The exploration code is on branch VAGOV-TEAM-89850-91349-form-flow and covers this spike in addition to another spike examining the feasibility of default paragraphs (steps).

Flow/Screenshots

Here are some screenshots of the custom flow:

1) Form basic info

2) Default paragraphs/steps added

3) Select additional paragraph/step type

4) Create additional paragraph/step

5) No more paragraphs/steps to add

6) Review and submit

Details

Custom Module

At the heart of the "solution" is the creation of a custom module va_gov_form_builder. This module enables us to define custom routing via a controller that directs each step of the process to an individual custom (sub-)form. Together, the different sub-forms constitute a full node-create/node-edit experience.

Architecture/Data Persistence

This spike code explored the pattern of creating/saving a node early and then editing it (new draft/revision) at each subsequent step. More specifically, the first "Basic Info" step results in the creation of a new Digital Form node, and then each subsequent step creates a new revision with the additional information from that step.

This pattern seems to work quite well. One key benefit is avoiding separate create and edit paths for every step along the way. After the first screen, everything is an "edit". This keeps things very clean from an architectural perspective.

One potential downside to this approach is the creation of many revisions. It will probably be necessary to explore defining what is actually a bona fide revision (certain amount of time since the last revision, perhaps) and then overwrite that revision with the step-by-step changes. That's just an idea at this point, but it feels reasonable at first thought.

Form Classes

As alluded to above, each step of the process is implemented as its own (sub-)form. That is to say, each step is implemented by a separate Class in va_gov_form_builder/src/Form/DigitalFormForm/. This works well, but this spike did not go so far as to properly design or implement an inheritance structure that would move some common functionality to a parent class. Each of these classes is going to share some common needs like tracking the current node being edited, and that functionality should likely be moved up to a shareable place.

Tying into existing forms/form configurations vs. "manually" rendering all form fields

One of the more important considerations when thinking about "ejecting" from the default experience is what that might mean for how things are tied into existing forms and form configurations.

For example:

1. Paragraphs: We see in screenshot 4 above the form for adding a Name and DOB paragraph. In the default experience, this form is rendered automatically from the fields defined on the paragraph. In this spike, those fields are rendered individually, by manually defining a form field for the fields we know are part of the paragraph. If a field were added to the paragraph, the form would not update automatically to reflect that. It is possible to render the entire paragraph form (the default paragraph form with all fields automatically included), but that would not allow any customization of that form itself, so it seems that we'll ultimately need to render fully custom paragraph forms as part of a fully custom node (multi-step) form.

   We would need to figure out the degree to which we'd try to loop through field settings vs. only showing form fields that we explicitly add to the paragraph form.

2. Node at large: The same issue exists for the node at large. Each step in the screenshots above reflects one subset of the fields on the node at large, and each of those steps is configured manually to show the fields that belong on that step. If another field were added, it would not render automatically. This makes sense, it seems, since there isn't (at present) any real way of knowing where the newly added field should render.

   One possibly related idea is the use of fieldsets. Perhaps each page/step of the step-by-step process could be grouped as a set of fields, and we could index into those sets based on the step "number". There might be multiple fieldsets on a given page, though, which could complicate matters. This is, again, just a thought, but, overall, this feels like an important question to consider.

Risk

There are two types of risk to consider here:

1. Risk of bugs/defects/breaking other things: This risk feels relatively low. In fact, this approach feels like it would introduce less risk than the approach we explored last sprint around a "partially custom" paragraph editing experience. Ejecting fully would afford us complete control, which would remove many vectors for risk around unknown moving parts or trying to force our desired behavior into a previously defined flow. That is to say, if we aren't going to go with the complete OOB experience, a fully custom approach feels like the path with the less risk than a partially custom approach.

2. Risk of poor ROI/heavy engineering lift: Now, there is certainly the reality that a fully custom approach will take more time and cost more. Again, though, this is to be contrasted against the alternative of going with the OOB experience. As soon as we talk about this vs. a partially custom experience, it's very hard to say. The risk discussed above paves the way for additional risk in unknowns, and that could lead to some work that is very difficult to estimate.

Summary/Decision

This spike was very promising with regard to the doors it opens for creating a customized UX. There are some technical details that need to be determined, but they seem at this point to boil down to decisions rather than obstacles. We should be able to move ahead with a plan of implementing our own customized user flow for creating/editing Digital Forms.