SPIKE: Evaluate Drupal's Webform module for viability

[ryguyk]

Issue

Need to investigate and evaluate the Webform ecosystem of modules for viability.

• How extensible is it?
• Can additional custom or composite elements be added easily?
• Can it support the VA's Chapter Page Element paradigm? (Comes with Page & Element OOTB)
• What does decoupled output look like for JSON:API / GRAPHQL / custom REST?
• How much massaging needed to align with RJSF used by VA Forms?
• investigate integrations with Workbench Moderation / VA sections
• can Webforms have revisions?
• How granular are permissions related to Webform?
• Can we support conditional fields in any way?

Additional Info

Helpful links: https://www.drupal.org/project/webform https://www.drupal.org/project/webform_rest https://www.drupal.org/project/webform_jsonapi https://www.drupal.org/project/webform_jsonschema

https://design.va.gov/patterns/ https://depo-platform-documentation.scrollhelp.site/developer-docs/va-forms-library-form-config-options https://depo-platform-documentation.scrollhelp.site/developer-docs/va-forms-library-about-schema-and-uischema

[tjheffner]

Webform notes

Module: https://www.drupal.org/project/webform

install steps:

in the va.gov-cms repo:

ddev start
ddev composer require drupal/webform
ddev drush en webform

visit https://va-gov-cms.ddev.site/admin/structure/webform

needed to enable webform_ui module for easy drag & drop editor interface. Also enabled webform_templates to allow for re-using existing forms. Both of these are sub-modules of webform and do not require additional installs.

initial thoughts

many features are probably not needed and/or should be hidden for majority of users.

any submission-related functionality is not needed for us, as submissions will never be coming back to drupal, but sent off to the appropriate endpoints. So we can ignore (hide/disable) most submission-related functionality and focus strictly on the form-building aspect. We will need to create some custom form fields for the submission endpoint, probably on a per form basis.

likely need at least 2 roles for permissions. i suggest form_builder are users who can create a form. a higher role (form_admin?) would be able to manage form templates & re-usable option groups (states, service branches, etc), global configuration, etc.

HIDE THESE GLOBAL TABS FOR ALL USERS: submissions, add-ons, help

customization + extension

a lot of individual form options can probably be hidden.

custom form elements & composites can be created easily via custom code. examples: docroot/modules/contrib/webform/modules/webform_example_element docroot/modules/contrib/webform/modules/webform_example_composite

basically need to create the Element and the Plugin. Element is a WebformElement which is a wrapper around a standard Drupal FormElement. Custom options will then show up in the UI as available options to construct a form.

look at using webform_simplify to hide certain options, or as an example for custom code: https://git.drupalcode.org/project/webform_simplify/-/tree/1.x/src?ref_type=heads this module hides portions of the webform ui based on permissions

Chapters

OOTB, webform provides a number of standard Elements as well as a Page type container, for making wizard-esque forms. The VA takes this wizard concept one further, with Chapters containing Pages. So a form can be a nested multi-step wizard. Not the greatest experience, but a reality of some of these very long forms.

By default, Pages provided by webform cannot be nested. So a form at most is one "Chapter". May be able to create an additional custom container element that allows for nesting based on the module's provided WebformWizardPage.php

form versioning + access control

no form revisions by default. webforms are stored as config and not as content, so revisions and moderation workflow do not function in the same way out of the box. it looks like https://www.drupal.org/project/config_entity_revisions may provide a way to create revisions for configuration-based entities.

webforms have a very permissive Categories field, but we would probably want to restrict that (or use the existing drupal section-based administration instead). this would involve custom code. We should be able to base a custom permission on webform_simplify examples for "Edit forms in my section" or something similar.

exposing form data

OOTB, the json:api endpoints for webforms are useless. to surface form data in a usable way for the frontend, will need to enable these modules and compare:

webform_rest - provides custom endpoints to retrieve form structure webform_jsonapi - to integrate with JSON:API, next-build webform_jsonschema - to output webforms in compatible format with RJSF, what the VA currently uses to build forms. some caveats around multivalue and conditional fields

webform_rest

enabling this module surfaces additional options as REST resources here: https://va-gov-cms.ddev.site/admin/config/services/rest

enabling the Webform Elements and Webform Fields resources fails at first, seems to be an error with some module handler under the hood. https://www.drupal.org/project/restui/issues/3327681 attempting to add this patch

After adding the patch, I was able to enable both Webform REST resources. endpoints:

• /webform_rest/{webform_id}/elements?_format=json

the Elements endpoint doesn't appear to populate anything for me.

• /webform_rest/{webform_id}/fields?_format=json

the Fields endpoint returns the structure, fields, and options for a given form

{
  "introduction_page": {
    "#type": "webform_wizard_page",
    "#title": "Introduction page",
    "address": {
      "#type": "webform_address",
      "#title": "Address",
      "#webform": "test",
      "#webform_id": "test--address",
      "#webform_key": "address",
      "#webform_parent_key": "introduction_page",
      "#webform_parent_flexbox": false,
      "#webform_depth": 1,
      "#webform_children": [],
      "#webform_multiple": false,
      "#webform_composite": true,
      "#webform_parents": [
        "introduction_page",
        "address"
      ],
      "#admin_title": "Address",
      "#webform_plugin_id": "webform_address",
      "#webform_composite_elements": {
        "address": {
          "#type": "textfield",
          "#title": "Address",
          "#admin_title": "Address",
          "#webform_composite_id": "test--address--address",
          "#webform_composite_key": "address__address",
          "#webform_composite_parent_key": "address"
        },
        "address_2": {
          "#type": "textfield",
          "#title": "Address 2",
          "#admin_title": "Address 2",
          "#webform_composite_id": "test--address--address_2",
          "#webform_composite_key": "address__address_2",
          "#webform_composite_parent_key": "address"
        },
        "city": {
          "#type": "textfield",
          "#title": "City/Town",
          "#admin_title": "City/Town",
          "#webform_composite_id": "test--address--city",
          "#webform_composite_key": "address__city",
          "#webform_composite_parent_key": "address"
        },
        "state_province": {
          "#type": "select",
          "#title": "State/Province",
          "#options": {
            "Alabama": "Alabama",
            ...
            "Yukon": "Yukon"
          },
          "#admin_title": "State/Province",
          "#webform_composite_id": "test--address--state_province",
          "#webform_composite_key": "address__state_province",
          "#webform_composite_parent_key": "address"
        },
        "postal_code": {
          "#type": "textfield",
          "#title": "ZIP/Postal Code",
          "#admin_title": "ZIP/Postal Code",
          "#webform_composite_id": "test--address--postal_code",
          "#webform_composite_key": "address__postal_code",
          "#webform_composite_parent_key": "address"
        },
        "country": {
          "#type": "select",
          "#title": "Country",
          "#options": {
            "Afghanistan": "Afghanistan",
            ...
            "Zimbabwe": "Zimbabwe",
          },
          "#admin_title": "Country",
          "#webform_composite_id": "test--address--country",
          "#webform_composite_key": "address__country",
          "#webform_composite_parent_key": "address"
        }
      }
    },
    "#webform": "test",
    "#webform_id": "test--introduction_page",
    "#webform_key": "introduction_page",
    "#webform_parent_key": "",
    "#webform_parent_flexbox": false,
    "#webform_depth": 0,
    "#webform_children": [],
    "#webform_multiple": false,
    "#webform_composite": false,
    "#webform_parents": [
      "introduction_page"
    ],
    "#admin_title": "Introduction page",
    "#webform_plugin_id": "webform_wizard_page"
  }
}

webform_jsonapi

https://www.drupal.org/project/webform_jsonapi this module appears to have been removed, actually.

ddev composer require drupal/webform_jsonapi
Could not find package drupal/webform_jsonapi.
Pick one of these or leave empty to abort:
  [0] drupal/react_webform_backend
  [1] drupal/decoupled_kit
 >

webform_jsonschema

this module provides an additional REST endpoint. still needs the restui patch from the webform_rest section above. additional endpoint: /webform_jsonschema/{webform_id}?_format=webform_jsonschema

this outputs data in json object with schema and ui properties, closer to what the VA form system currently uses (but potentially not exactly the same)

{
  "schema": {
    "title": "test",
    "type": "object",
    "properties": {
      "introduction_page": {
        "title": "Introduction page",
        "is_wrapper_element": true,
        "type": "object",
        "properties": {
          "address": {
            "title": "Address",
            "is_composite_element": true,
            "is_wrapper_element": true,
            "type": "object",
            "properties": {
              "address": {
                "title": "Address",
                "type": "string"
              },
              "address_2": {
                "title": "Address 2",
                "type": "string"
              },
              "city": {
                "title": "City/Town",
                "type": "string"
              },
              "state_province": {
                "title": "State/Province",
                "type": "string",
                "anyOf": [
                  {
                    "enum": [
                      "Alabama"
                    ],
                    "title": "Alabama"
                  },
                  ...
                ],
                "uniqueItems": true
              },
              "postal_code": {
                "title": "ZIP/Postal Code",
                "type": "string"
              },
              "country": {
                "title": "Country",
                "type": "string",
                "anyOf": [
                  {
                    "enum": [
                      "Afghanistan"
                    ],
                    "title": "Afghanistan"
                  },
                  ...
                ],
                "uniqueItems": true
              }
            }
          }
        }
      }
    }
  },
  "ui": {
    "introduction_page": {
      "address": {
        "ui:order": [
          "address",
          "address_2",
          "city",
          "state_province",
          "postal_code",
          "country"
        ]
      },
      "ui:order": [
        "address"
      ]
    },
    "ui:order": [
      "introduction_page"
    ],
    "webform:generalValidationErrorMessage": "A form validation error occurred. Please check the values you have entered.",
    "webform:generalSubmissionErrorMessage": "There was an error submitting webform."
  },
  "buttons": [],
  "data": [],
  "csrfToken": "tU0loO4670C0SD1_ezNiE1pB5a60pXPcfcZ6Yh8Av4k"
}

The webform_jsonschema seems promising. Another option is rolling our own REST (or JSON:API) endpoint without a third party module.

Next I will attempt to create a custom element for "Chapters" and then use that in a re-usable form template.

[tjheffner]

Creating custom & composite elements is fairly straightforward. You need two files in the src/ of a custom module:

• Plugin/WebformElement/<yourElement>
• Element/<yourElement>

One of these defines the Webform Plugin and the other handles the FormElement that the WebformElement is wrapping.

I used this approach to verify that:

• we can easily additional custom or composite elements ✅
• can support the VA's Chapter Page Element paradigm ✅
• these custom elements still output in the jsonschema endpoint ✅
• conditional fields are supported. It doesn't appear to support conditional labels OOTB

see this PR for more details on writing custom elements: https://github.com/department-of-veterans-affairs/va.gov-cms/pull/18248

---

Next, I need to dig into the ability for webforms to have revisions and how that might be managed, as they are configuration entities.. not content entities. This is potentially a major roadblock to using this approach. Storing forms as configuration entities means there are no revisions or moderation states by default, and that webforms created in the prod cms are not stored in a database (or captured in code).

[tjheffner]

Testing webform revisions has proven harder than expected.

First, I tried to install & enable config_entity_revisions and its webform_revisions submodule. I was able to add webforms as an entity type within the existing Editorial Workflow in the CMS (Draft, In review, Approved, Published, Archived), however when attempting to make edits to a webform errors are thrown. No issues in the queue looked immediately relevant for a fix. This approach may be viable for creating webform revisions, but would need to solve these errors to be sure.

Second, attempting an alternate approach to revision management, via the module webform_node. This module provides a Webform content type which allows webforms to be integrated into a website as nodes via an entity reference field. This means the nodes would have revision workflows and section-based access management, and the webforms would still be managed separately.

I enabled webform_node and attempted to create a new Webform node. This errored, as the VA CMS looks for field_last_saved_by_an_editor on nodes. I added this field to the provided Webform content type. I can now visit the Webform node edit page, and attach an already-created webform via an entity reference field. I cannot save the node however, and no errors are thrown in the browser console or logged by Drupal... the submit button simply stops working. I have verified that other content types are still able to save as expected, so it is only an issue with this Webform content type provided by the module. Neat.

I then attempted to add a new entity reference field to the content type, referencing webform config. I deleted the reference field that came with the module. Node can still not be saved.

I then attempted to add a webform field to an existing content type that could be saved (e.g. Story). This node was able to be saved...but the webform field throws an error in the jsonapi response. So not perfect. Updating the JSON:API resource override for the newly created field, JSON:API no longer throws an error, but the field data is not displaying the referenced form. Some issues.

So, none of the out-of-the-box solutions for revisions or content moderation workflows work perfectly as expected. Some custom code, or patches to these modules, will be required to handle these particular requirements.

[tjheffner]

final thoughts

With a little configuration, the Webform module ecosystem provides a decent OOTB experience and feature set. It does require a number of enabled modules, at minimum:

webform webform_ui webform_templates webform_rest webform_jsonschema a patch for the restui module

webform_simplify can also be added to jumpstart hiding ui elements based on role permissions.

Speaking of permissions, we would likely need at least 2 form-related roles for building & managing digitized forms.

It seems easy enough to create custom elements or field combinations we'd like editors to re-use.

webform_jsonschema provides a good base output for webforms in a way that's compatible with react-jsonschema-form

Revision handling of webforms is the biggest unknown, but there are a couple different options:

• Fix the errors in config_entity_revisions
• Fix the errors in webform_node
• Roll our own revision integration for webforms

[ryguyk]

Decision

We will NOT move forward with the Webform module at this time.

• First and foremost, it seems very likely that using the Webform module would present higher risk:
  • Webforms are stored as config rather than content, and this would represent a significant paradigm shift in how we are handling editor-managed data (content). This change seems significant.
  • Rather than use existing tooling for fetching content via GraphQL/JSON:API, we would need to build out some new tooling for fetching configuration.
  • The config-vs-content reality also presents some challenges around editorial workflow.
  • Managing revisions becomes a significant lift (if it would be possible at all).
  • The solutions we have in place for managing various editorial states (draft, in review, approved, published, archived) would not be nearly as easy (if possible at all) to adopt.
  • Using Webform would require a lot of new modules
  • This would probably amount to 5+ new modules. Going down the alternative path - Paragraphs - would not immediately lead to any new modules (this could change down the road, but would almost certainly be a smaller footprint regardless).
• Ultimately, it feels like the Webform module does not cleanly capture our use case.
  • We are decoupled, with a feature-rich front-end Forms Library already present. Webform has many settings around display of fields, etc. that simply would not apply to us.
  • We might go as far as to say, in fact, that we aren't looking to build forms, per se, but rather a configuration that represents a form that will be built by another system. As such, it seems like the Webform module does not really capture our use case.