Unexpected problem processing entity when form is updated to allow entity creation

[lognaturel]

Problem description

When a form definition didn't enable entity creation and then is updated to enable it, Central still rejects the create on entity reprocessing.

URL of the page

https://test.getodk.cloud/#/projects/149/forms/create_update_beneficiary/submissions/uuid:611d7172-adb1-4e84-9799-fd09c199aa24

Steps to reproduce the problem

Use a form like https://docs.google.com/spreadsheets/d/1R3Ja6hOjjntE42HaYQMf0MLITMSsIfOlbb60Fe9l_kM/edit#gid=1323131743 that attempts to create or update entities. Omit createif and update if on the entities sheet

Make a submission that attempts to create an entity, see an error on the submission feed as expected.

Update the form to add create_if and update_if

Edit the submission in Enketo, don't make data changes, submit again

The entity processing happens again but now fails because the submission attempts to create an entity but the form doesn't allow it.

Screenshot

Expected behavior

Entity re-processing succeeds

Central version shown in version.txt

versions:
9a9ec7c034de07e990c9c408ca768326e2154c96 (v2023.5.1-10-g9a9ec7c)
+c25bc15087ca045496d3bdbc806ca0752b955216 client (v2023.5.0-64-gc25bc150)
+cfe46c97e6e38d8e551ce253ba47d0b378d19352 server (v2023.5.0-56-gcfe46c97)

Browser version

Around when did you see the problem (in UTC)?

Other notes (if any)

I suspect this is because Enketo uses the latest form definition but Central uses the form definition that was used to create the initial submission.

[matthew-white]

The XML of the form definition can be found on the Versions page, but here are some quick links for the submission XML:

• Initial submission
• Edited submission

One thing that stands out to me is that both submission versions specify version="20240425164246", which is the first version of the form. The first version only permits entity update, not entity create. I also thought that Enketo uses the latest form definition, but then why isn't the version string different for the edited submission? If the version string matched the latest form definition, then I think the submission would be processed successfully.

Another thing that confused me at first is the initial error message: "Required parameter baseVersion missing." That happens because the initial submission specifies update="1" baseVersion="". It's processed as an entity update, but it's invalid because baseVersion is empty.

I suspect … Central uses the form definition that was used to create the initial submission.

It looks like this is true for submission parsing, as well as for checking whether the specified entity action is permitted. For checking the action, we could check against the actions listed in the current form definition if that's better.

[matthew-white]

I suspect … Central uses the form definition that was used to create the initial submission.

It looks like this is true …

Sorry, that's not quite right! To be precise, I think it uses the form definition identified by the version string in the submission definition.

[lognaturel]

I also thought that Enketo uses the latest form definition, but then why isn't the version string different for the edited submission?

Version string population is entirely based on client implementation so my guess is that Enketo just doesn’t ever change it! Currently Collect never uses an updated form version to open an existing submission so I imagine it has the same behavior.

So it sounds like this should actually be an Enketo issue!