[CONSIDER] Extend VA Service synonyms/keywords text fields to >255chars

[davidmpickett]

User Story or Problem Statement

As a consumer of Content API I need term lists to able to contain an extensive list of synonyms So that I can better match user input to the VA Service that matches their need

Common Conditions

The Common Conditions fields are lists of keywords that help users know they are looking at the right service. Right now they are a single comma, separated text field, with a 255 char max:

• Term: Cardiology
• Common conditions: tinnitus, vertigo, hearing loss, vestibular conditions, swallowing conditions

Patient-friendly Name

The Patient-friendly Name fields aren't the exact same use case as Common Conditions. They aren't perfectly formatted as lowercase comma separated lists of words. Some times they are a single synonymic phrase

• Term: Vocational rehabilitation and employment programs
• Patient-friendly name: Job and career training programs

And when they do take the form of lists of words, they are usually formatted more like a sentence fragment:

• Term: Complementary and integrative health
• Patient-friendly name: Alternative medicine, acupuncture, yoga, and meditation

They also may include acronyms or initialisms, which is technically a different type of relationship than a synonym and retaining case might be more important.

• Term: Physical therapy, occupational therapy and kinesiotherapy
• Patient-friendly name: PT, OT, KT

The inconsistency in formatting means that Patient-friendly Name will be even harder for API consumers to decompose. Consider how a script that treats " and " the same as it treats a comma might break down the examples above:

• Alternative medicine, acupuncture, yoga, and meditation

  • Alternative medicine
  • acupuncture
  • yoga
  • meditation

• Job and career training programs

  • job
  • career training programs

Steps for implementation

Increase field size via _va_gov_db_change_text_field_max_length() in va_gov_db/va_gov_db.install

Acceptance Criteria

• [ ] Common name and Patient-friendly name Fields are audited for whether we are approaching 255char max
• [ ] Field character limits are extended from 255 to (TBD?)
• [ ] Update tests as required
• [ ] Confirm Lighthouse services push isn't affected & fix if so
• [ ] MERGE TO INTEGRATION BRANCH: shouldn't go to main til FE work is complete (still true?)
• [ ] Jordan to provide help text
• [ ] Requires Content Model Documentation
• [ ] Requires design review

[xiongjaneg]

@xiongjaneg stub out FE ticket to follow this one

[jilladams]

From scrum today, with Christian / Eli: In theory, it may actually be easier to parse the text field. Because then you can check the full field data and flag any match / partial match. Parsing an array may actually be more complex.

KISS files are generated to do a GraphQL query on the taxonomy/terms, or a way to create a view that can be downloaded as a CSV. Will do it the same as the SPIKE results suggested: download all the taxonomy terms including their common names, etc., and is done in content-build. Even if Drupal delivered an array, the front-end would likely have to reassemble all those items into a grouping that can be searched easily. So: more structure in the data isn't necessarily helpful, and may require more nested calls.

Also: this taxonomy is highly structured and has a limited editor base. If we were opening this up to all VAMC editors, we could make an argument for arrays as easier to manage. But: given the editor base, that's not maybe the most valid argument.

@omahane to think harder about this alongside #14993 and we will revisit next week in scrum to finalize decision.

[jilladams]

Reviewed with Michelle, concluded the UX here isn't necessary for Admins editing the taxonomy unless it's a win in terms of data structure for the front end. Since it's not, @mmiddaugh to clarify with DaveC re: if she's missing anything. If not, we'll close as no-op.

[jilladams]

Noting: if we do make this change, data will still need to be squashed back down for FE search. There's a package, Fuse, available to the FE to parse the more structured data if we go this route. We're reluctant to add packages to support single use functionality, but could if we need to.

Christian also flagged text fields ( Common Names, Patient Friendly Names) is capped at 255 chars, which might be a concern.

[jilladams]

We may be able to up the character count on the text fields, if that's the main concern. Maybe. @omahane to verify if that can be done in the schema.

[jilladams]

@omahane Michelle also thought about: we push services to LH, and will need to make sure we don't affect the success of that push if we change this, or will need to account for it.

[omahane]

Regarding the services push to Lighthouse, we don't push the common conditions.

[omahane]

Regarding changing the schema to increase the field size: _va_gov_db_change_text_field_max_length() is in va_gov_db/va_gov_db.install to do just that.

[jilladams]

Final answer from scrum:

• No solid reason to switch to data arrays. We won't.
• We can extend past 255. We only care about doing that if we need to. Right now it's forward looking, in case some of our services generate lots and lots of synonyms.
• This is not a priority currently.

[davidmpickett]

Quick analysis of Common Conditions & Patient Friendly Names character counts.

Common Conditions

• Average 72 characters
• Highest current example = 201 characters

Patient-Friendly Names

• Average 31 characters
• Highest current example =79 characters

https://dvagov.sharepoint.com/:x:/s/SitewideFacilitiesTeam/ERKcUmY00NlJtCQWzo2TSZEBu1AeJoyRE7rhZJgO9MpqYg?e=fpLSks

[jilladams]

2 things:

1. I updated ticket body to reflect that this is now an optional backlog ticket for converting to >255chars for these 2 fields. The old ticket body is below, in case we need it bc of question 2.
2. The Patient friendly name notes in ticket body are something we didn't really discuss when we decided not to move to data arrays. Those are sentence-y-er and include commas in their own right, which makes it harder to explode on commas for the FE. @eselkin could you read through the Patient Friendly Name notes Dave included on that in ticket body and just triple confirm that we are ok leaving things as is? (cc @omahane )

Old ticket definition notes

Ruled out: Data arrays

Facilities team in April 2024 considered converting these fields to become data arrays but ruled out this approach. Notes below are from that proposal, for reference. Data arrays, e.g.

• Term: Cardiology
• Common conditions:
  • tinnitus
  • vertigo
  • hearing loss
  • vestibular conditions
  • swallowing conditions

This was ruled out for a few reasons (in comments below):

• When data is delivered the FE, it is flattened again to become a searchable object anyway
• The Editor UX is a limited concern since the VA Services taxonomy has a limited set of editors with permissions to modify
• The change wasn't required to enable extended search functionality in the front-end

Potential path for array implementation

Copy what Public Websites did for field_va_benefit_plain_name on the VA Benefit Taxonomy

Tentacles

• FE displays common conditions on VAMC accordions. This change will break FE and should be staged to an integration branch until FE is ready to accommodate the change.
• Field data shows in the CMS FE

Bonus side effect

If we add a new field and migrate existing data into it, we could also clean up some of the legacy naming conventions

If we did this, ACs would've been

• Field types are changed from text to (multi-value)
• Migrate existing field data to the new fields
• Update tests as required
• Confirm Lighthouse services push isn't affected & fix if so
• MERGE TO INTEGRATION BRANCH: shouldn't go to main til FE work is complete
• Jordan to provide help text

[jilladams]

Or maybe we could just separate them on something besides commas?

[mmiddaugh]

Per Dave P's comment, we have no examples of insufficient character counts Sending this to the icebox for now

[Agile6MSkinner]

Going to remove from epic.