Persist Data Across Sessions

[engineering-issue-sync-app[bot]]

Value Proposition Statement

Save time extracting data from complex JSON responses and entering data when playing your process

User Problem

• Handling process data using FEEL expressions often requires multiple guess-and-check cycles to get right because users rarely use the "example data" feature. Therefore, the Modeler doesn't know the expected data structure.
• Form autofill responses are saved to local storage, meaning they:
  • cannot be shared between users, reducing collaboration
  • can be lost when clearing the cache or changing browsers, resulting in rework
  • cannot be used elsewhere in the C8 stack, decreasing reusability and consistent UX

Release Notes Don't Repeat Yourself (DRY). Play now supports an end-to-end journey for saving and reusing example data.

• Save example data to the BPMN diagram while playing your process so anyone can use it
• Any user can use this example data in their sessions, eliminating the need for tedious form-fills or copy-pasting
• Connector example data is now available to help define output variables or mock connectors more easily

User Stories

Saving Example Data

• [x] (M) Local Vars: As a developer, I want to save local Play variables to the element that created it so I can reuse the data in context
• [x] (M) Global Vars: As a developer, I want to save global Play variables to the element that created it so I can reuse the data in context
• [ ] (C) Migration: As a process developer, I want to migrate my local storage form fills to example data so it's persisted
• [x] (S) Lists of Variables: As a process developer, I want to add multiple different variables to an element's example data so I can reuse them
• [x] (S) Overwrite: When I try to add a variable that already exists, I want it to be replaced so I can update my example data
• [x] (M) When I save example data, I want it to be a validated JSON object so I can use it in Play
  • Resolved by https://github.com/camunda/camunda-modeler/issues/4220
• [ ] (M) Access Control: When I don't have write access to the BPMN diagram, I want to be prevented from saving data so I don't introduce unwanted changes
• [ ] #2390
• [ ] (C) Connector Template Schema: As a connector developer, I want to save example data to the template so I have a data schema that's ready to use
• ~(W) Form Builder: When I build a form, I want persisted data in the "Form Input" section so I don't have to re-enter it after the page refreshes (potential followup for HTO team)~
• ~(W) Regression Checking: When I change my form, I want to be warned if my example data is invalidated so I can prevent regressions (potential followup for HTO team)~

Using Example Data:

• [x] (M) When completing a start event, I want to quickly use example data so I can efficiently run my process
• [x] (M) When completing a user task form, I want to quickly use example data so I can efficiently run my process
• [x] (M) When completing a service task, I want to quickly use example data so I can efficiently run my process
• [ ] (M) When completing a connector, I want to quickly use example data so I can efficiently run my process
• [x] (C) When sending a message, I want to quickly use example data so I can efficiently run my process
• [x] (C) When receiving a message, I want to quickly use example data so I can efficiently run my process

Adoption:

• [x] (S) Awareness: As a user who has never used this feature, I want to be alerted to its existence when it would be valuable so I can benefit from it
• [ ] (S) Onboarding: As a user who has never used this feature, I want an explanation of its value and user journey so I know what to do

Efficiency Improvements - requires user testing

• ~(C) Bulk Save: When I complete a process, I want to save all form fills so I can easily replay my process~
• ~(S) General Forms: As a form developer, I want to save my form fill so I can reuse it later~
• [ ] (C) User Task Forms: When completing a form, I want to save all outputs as a single action so I can efficiently save example data

Multiple Option Support - later iterations

• ~(W) Save Alternates: As a process developer, I want to add multiple options for the same variable so I can play different scenarios~
• ~(W) Select Alternates: When I open a modal to complete a Play task, I want to select which example to use so I can run the right scenario~

Implementation Notes

• The most critical modals to support are start events, user task forms, service tasks, and connectors. All other modals should be considered and decided to be in-scope on a case-by-case basis.
  • ⚠️ The form editor has a sample "Form Input" section, but it's not persisted and not synced to the "Example data" section. Saving form fills with the BPMN diagram does not apply when there are multiple processes, nor does it help in form editing.
• Example data is part of the BPMN XML and has some advantages
  • It is checked into source control
  • Changes can be tracked and reviewed
  • Any user can take advantage of it
  • It's portable across development stages (e.g. dev, int, stage)
  • Called processes are supported by default
• Being able to reference example data might be a first step towards Process Testing MVP
  • When selecting a response, a plaintext title or description is likely the most vital information
  • Created by and created date could be secondary information
  • Developers continue to ask whether they can save their sessions and replay them automatically as in a test suite. We should build towards the user journey of 'play a process, save tests from the process, run test suite"
  • The data should be formatted in a way that facilitates testing
  • Ensuring that saved data can be checked into or retrieved from source control is an essential first step
  • We should not try to achieve that vision with this epic. This is the first step. We need it to be proven valuable to justify further investment
• There are 2 broad solution approaches
  1. Blindly append to process example data, have output variables pull from global scope example data
     • Simpler, but poorer UX
  2. Map global variables to BPMN elements (preferred)
     • Works for service tasks, not for connectors
     • Connectors don't have example data
     • Enables "complete with example data"
• Connectors use the connector runtime, not Zeebe. Output mappings will not be executed when completing with variables.
• Mapping during execution only works in Play with modals. Mapping after execution is harder to map to an element
  • Connector responses cannot be used because there's no modal when they are executed

^{:robot: This issue is automatically synced from: source}