Discuss: 'Background'-based tests

[plocket]

In combination with human languages

Proposed: Each file contains only one 'story' set up as a cucumber Background. Each Scenario in it just sets a language for the test. We handle skipping or testing languages with cucumber tags.

'Story' here means one set of user data (has 3 children, lives in an apartment, lives out of state, etc.) Usually this might be called a 'scenario', but cucumber uses that term for specific functionality that is also part of this discussion.

Pros

1. More fine-grained developer control over languages for each 'story'

Cons

1. See questions

Questions

1. Would a developer have to list all these languages in each test? If so, if a new language gets added, it would need to be added manually to every test as opposed to being automatically generated. If not, what is the advantage of this method over the current method? If it's to do with adding tags, tags can be added by code that auto-generates the language tests as well.
2. Developers have expressed a desire to give some set of variables the same values for a few different 'stories', each of which would vary some specific variable values. Would they still be able to do that with this setup?

[plocket]

I think Background might be better to [make use of] its current functionality - running some steps before each Scenario in a file. [For example, allow dev to set the foundation of a story table that they can build on later. I'm not sure how to then differentiate it from tables they add later, or to let them make a table that's not based on that base story table. It's complicated. We can start with just a base/foundational table, though.]

[plocket]

The pre-release for the story table Background step is out. Currently for v3 on branch https://github.com/SuffolkLITLab/ALKiln/tree/75_background_table_v3. Here are some general instructions for trying it out:

To start with, we're trying pre-release version 3.3.0-background-table.2. If you're not using a GitHub secret:

           ALKILN_VERSION: "3.3.0-background-table.2"

Get ready

Depends on the setup. Basically need to make sure that we can pull in a pre-release version of the lib.

If the repo has its own package.json, just update version of docassemble-cucumber to the pre-release version.

If the repo only has the workflow file, no package.json:

• In the workflow file, add ALKILN_VERSION as input to the step that uses suffolkLITLab/ALKiln@releases/v3.
• Either put in the version you want inside the brackets, as shown at the top of this comment:

         ALKILN_VERSION: "x.x.x..."

  or

• Add ALKILN_VERSION GitHub secret with the value of the pre-release version you want and put secrets.ALKILN_VERSION inside the brackets:

         ALKILN_VERSION: "${{ secrets.ALKILN_VERSION }}"

Write the tests

You'll need to know the id of the last page you want to get to for each Scenario.

• To the beginning of your test file and add

  Background:
  Given I always have this data:

• Add the story table under there Example:

  Background:
  Given I always have this data:
  | var | value | trigger |
  | users[0].name.first | Uli | users[0].name.first |
  | users[0].name.last | User1 | users[0].name.first |

• In the first scenario, make sure to wait at the start to give the package module extra time to reload the server
• In each Scenario, start by going to the right address
• Add the step:

  And I get to "final question id" with the background data

  Example:

  Scenario: index.yml runs
  And the maximum seconds for each Step in this Scenario is 800
  And I wait 800 seconds
  Given I start the interview at "index.yml"
  And I get to "final question id" with the background data

Edit: Example here while the branch is alive: https://github.com/SuffolkLITLab/ALKiln/blob/75_background_table_v3/tests/features/observation_steps.feature#L6-L17 then https://github.com/SuffolkLITLab/ALKiln/blob/75_background_table_v3/tests/features/observation_steps.feature#L45-L48

[BryceStevenWilley]

I've rebased this to be on version 4 now, still leaving the old branch in case there's still something useful.

Can't make a pre-release personally, but I'll personally try to test it a little bit, and if I don't run into any issues, or confusing parts, I'll make a PR with this code.