Bug: CMS migrator only overwriting the last row in the Revisions database on Facilities products

[xiongjaneg]

Background

Public Websites discovered a migration problem that may also affect Facilities migrations: the nightly CMS migration for Forms is not correctly creating Revisions on Forms nodes.

The CMS migrator is always only overwriting the last row in the Revisions database, rather than writing a new row.

The new CMS Migrator revision squashes the most recent previous Revision into it (regardless whether previous is an Editor generated or a CMS Migrator revision). The new Revision contains both sets of changes.

This could snowball: every successive CMS migrator revision will squash the Revision before it. Over time, this could end up with a single Revision that represents several squashes, if only nightly changes are being made, if that makes sense. (Meaning; if a node gets edited 5 times in a day, only the last revision will get squashed into the nightly migration. But if a node is edited once a day, every day's edit will get rolled up into the next nightly CMS migration revision.)

The Forms migration is also no longer correctly applying all the Flags we expect to be applied to Forms. (Specifically our Changed Title flag is no longer being correctly applied.)

PW is going to work with CAIA since they're regularly editing, just to see if we can confirm that edited content is successfully capturing all the expected revisions.

There's also a chance this will trace back to Drupal core and become a CMS team problem.

Flags

Public Websites also has an instance of Flags not being correctly applied during migration. While checking migrations, it would be wise to test flags and attempt to verify if flags are being applied as expected.

Helpful links

• Slack thread on this issue
• PW Spike that discovered the issue: #17450
• PW: VA Forms not creating new Revisions during migration #17567
• PW: VA Forms not flagging Title changes during migration #17569
• Facilities recently updated by migration, if helpful for exploration
  • Hopi Vet Center Outstation

Problem statement

Facilities needs to know if CMS migrator is not correctly creating revisions on Facilities products nodes.

Acceptance Criteria

• [x] Facilities products nodes are reviewed to determine if CMS migrator is overwriting revisions.
• [x] When migrations make updates a new revision is created and only the updates made by the migrate are included.
• [x] If there is something that should trigger a flag as part of the migration's update then the flag is set.

[jilladams]

Recording of Daniel showing the problem description / demo between prod and Tugboat:

• https://us06web.zoom.us/rec/share/Yp8kMnr5VKjGbp1O70CCRevLVT2XsHw_Vzaz99Em7UMKN1dJHt-tDXkrlqr0GS4W.F9of9n7JHTBWB0D5?startTime=1711045894000
• Passcode: 7fdqnUq#

[Becapa]

Migrator made changes to the East Los Angeles Vet Center node on 03/19. It also made changes on 02/02 at 8:00am. 'The Last Saved by an Editor' is set to 02/02 at 11:36am, but there are no revisions in between the changes from the CMS Migrator:

Revisions Tab For the Node

When comparing the last revision with its previous revision there are Hours changes, but there are also Local Spotlight changes that probably shouldn't have been coming from the CMS Migrator:

Comparison of latest revision

[xiongjaneg]

There's a known defect with Vet Center last saved by an editor #15124

[davidmpickett]

Also I just found a Tugboat that built a few days ago and checked Hopi. It has a revision dated March 13 that seems to be overridden by the March 19th one

[Becapa]

So I think this is something that happened in Drupal core. It looks like in 10.2 they made it so that when an entity is updating from a migrate there is an isSyncing flag that is being set. I guess this keeps the migrate from being able to set the update as a new Revision (see https://www.drupal.org/node/3052115). There is a workaround that's documented here: https://www.drupal.org/node/3426397

[Becapa]

Plan for moving forward with code fixes:

• Clear up revision squashing:
  • Add the suggested workaround from the drupal issue (but in an EventSubscriber rather than a hook).
• Make it so flags are properly set during migrations:
  • Add a new property to the Flagger service that says whether or not a migration is currently running.
  • Add functions to toggle this property during the pre and post migration events.
  • Add the new Flagger property as one of the checks in flagFieldChanged() and setFlag().

[jilladams]

Proposal: since we're on a roll and have time in sprint, let's update this as the implementation, not just spike? Also ticket doesn't have points, so we've already spent prob 3-5, and let's add more for the rest? @Becapa any concerns about handling it that way? If not, I'll update ACs.

[Becapa]

@jilladams That sounds good to me!

[omahane]

I spent time on Friday and today making a view to make sure that facility name changes have not slipped by without our notice. The good news is that it appears none have happened since 10.2 came out. Do we want this view added to prod? @xiongjaneg

[xiongjaneg]

The more views the better? It sounds like it's been helpful for you so may be so again in the future. Thanks!

[omahane]

After a little more discovery, I found that we already have a view that does this: https://prod.cms.va.gov/admin/content/audit?title_op=contains&title=&type_op=in&type%5B%5D=nca_facility&type%5B%5D=health_care_local_facility&type%5B%5D=vba_facility&type%5B%5D=vet_center&type%5B%5D=vet_center_mobile_vet_center&type%5B%5D=vet_center_outstation&moderation_state_op=or&section=All

No need for an additional view.

[jilladams]

@omahane @dsasser any thoughts on how we might verify this to close it?

I took a look to see if Forms would make it easy. PR merged after 3/27 nightly migration. 3/28 nightly migration only modified one form from what I can see:

• All forms, only one with a 3/28 mod timestamp: https://prod.cms.va.gov/admin/content?title=&type=va_form&moderation_state=All&owner=All
• The modified form: https://prod.cms.va.gov/find-forms/about-form-29-541
• The revision: https://prod.cms.va.gov/node/5719/revisions/view/817206/818352/split_fields -- timestamp updated to reflect new revision date, and I can't tell that anything else is amiss on that node.
• CSV from that migration, with this form on line 95: va_forms_data (4).csv, where Revision date matches the new listed timestamp on the revision.

Not totally obvious to me that anything's wrong, but I'm not sure what would make me think that other than another stomped edit.

[jilladams]

End of sprint: discussed with Christian / Daniel. Forms looks ok. We'd like to monitor migrations for a little bit just to make sure nothing looks amiss before we declare success. Will keep ticket open for follow up in next sprint.

[omahane]

OK, we do have some strangeness going on, all circling around March 2, 2024, 7:28 am. (CT)

Updated date not matching the revision log

Content view (/admin/content)

Palm Bay VA Clinic was updated on April 8, 2024

Recent changes

No April 8 revision is shown.

Comparing revisions

The only difference is the addition of a health service.

VAMC facility health service creation

The time at which the health service was created is the same time at which /admin/content lists the facility as being updated. The facility is updated when a new service is created that is related to it by way of the corresponding entity reference.

I suspect that the creation of health services is somehow bypassing the normal process and is creating a new regression, not squashing revisions but using the same time as the last revision when creating a new one.

[jilladams]

Ah, ok. From forms perspective, we have evidence of the migrator making multiple revisions since we pushed this change on 3/27: https://prod.cms.va.gov/find-forms/about-form-28-10289 - has updates from 4/2 and 4/4 both from the migrator.

So: I think your notes belong in a new ticket, and we can call the "multiple revisions squashed to a single revision" bug fixed. Let me know if you agree, @omahane and I'll close and put together the new ticket.

[jilladams]

https://github.com/department-of-veterans-affairs/va.gov-cms/issues/17808 tracks the new report