Camunda Form Definition Version Not Incrementing on Change

piyushaswani55 commented 1 month ago

Environment (Required on creation)

Ubuntu/MacOs camunda-bpm-run-7.19.0 jdk-11 (Azul Zulu)

Description (Required on creation; please attach any relevant screenshots, stacktraces, log files, etc. to the ticket)

Camunda Form Definition Version Not Incrementing on Change and it always remains to 1 which results in below error

org.camunda.bpm.engine.ProcessEngineException: ENGINE-03109 Cannot resolve a unique Camunda Form definition for key 'camundaForm' because it exists for multiple tenants.

Steps to reproduce (Required on creation)

Deploy a BPMN resource with Camunda Task Form.

<userTask id="userTask" name="User Task" camunda:formRef="camundaForm" camunda:formRefBinding="latest">
  <outgoing>Flow_15mrist</outgoing>
</userTask>

Make changes to the camundaForm.form associated to the above userTask
A new version of camundaForm should be deployed

Observed Behavior (Required on creation)

The version of camundaForm stays 1 no matter how many modifications are made to the form and is never incremented

Expected behavior (Required on creation)

The version of the form should increase with every deployment of the form (as per the configuration)

Root Cause (Required on prioritization)

There is a small issue with the SQL query parameter due to which the result is always null and during deployment of the form changes there is no latestDefinition found, hence version is never incremented when newDefinition is persisted in the DB.

Solution Ideas

SQL query for selectLatestCamundaDefinitionByKeyAndTenantId in CamundaFormDefinition.xml has incorrect parameter due to which no result is returned, and form version is never incremented in the DB.

Current Query

<select id="selectLatestCamundaDefinitionByKeyAndTenantId" parameterType="map" resultMap="camundaFormDefinitionResultMap">
    select *
    from ${prefix}ACT_RE_CAMFORMDEF RES
    where KEY_ = #{camundaFormDefinitionKey}
          and TENANT_ID_ = #{tenantId}
          and VERSION_ = (
              select max(VERSION_) 
              from ${prefix}ACT_RE_CAMFORMDEF 
              where KEY_ = #{key} and TENANT_ID_ = #{tenantId})
  </select>

New Query key parameter in the above query should be updated to camundaFormDefinitionKey

<select id="selectLatestCamundaDefinitionByKeyAndTenantId" parameterType="map" resultMap="camundaFormDefinitionResultMap">
    select *
    from ${prefix}ACT_RE_CAMFORMDEF RES
    where KEY_ = #{camundaFormDefinitionKey}
          and TENANT_ID_ = #{tenantId}
          and VERSION_ = (
              select max(VERSION_) 
              from ${prefix}ACT_RE_CAMFORMDEF 
              where KEY_ = #{camundaFormDefinitionKey} and TENANT_ID_ = #{tenantId})
  </select>

Hints

Links

Breakdown

### Pull Requests
- [ ] https://github.com/camunda/camunda-bpm-platform/pull/4493

Dev2QA handover

[ ] Does this ticket need a QA test and the testing goals are not clear from the description? Add a Dev2QA handover comment

mboskamp commented 1 month ago

Hi @piyushaswani55,

Thanks for raising this bug report, including all that information, and for taking the time to investigate a root cause.

I am having trouble reproducing the error locally. Here is what I did:

Created a simple Camunda form
Created a simple user task referencing the form
Deployed both together in one deployment
Changed the form and redeployed only the form
In the DB, I can see two versions of the form deployment (see SQL snippet below)

When executing this

SELECT dep.ID_, dep.NAME_, form.ID_, form.REV_, form.VERSION_, form.DEPLOYMENT_ID_ FROM ACT_RE_DEPLOYMENT dep JOIN ACT_RE_CAMFORMDEF form ON dep.ID_ = form.DEPLOYMENT_ID_ WHERE form.KEY_ = 'camundaForm'

I get the following result:

ID_	NAME_	ID_	REV_	VERSION_	DEPLOYMENTID
9135df2e-48d7-11ef-b7aa-387c761673de	diagram_1	camundaForm:1:91496732-48d7-11ef-b7aa-387c761673de	1	1	9135df2e-48d7-11ef-b7aa-387c761673de
eaa97fe3-48d7-11ef-b7aa-387c761673de	form_1	camundaForm:2:eaa9ce05-48d7-11ef-b7aa-387c761673de	1	2	eaa97fe3-48d7-11ef-b7aa-387c761673de

Also, when looking at the ACT_RE_CAMFORMDEF table, I get this:

ID_	REV_	KEY_	VERSION_	DEPLOYMENTID	RESOURCENAME	TENANTID
camundaForm:1:91496732-48d7-11ef-b7aa-387c761673de	1	camundaForm	1	9135df2e-48d7-11ef-b7aa-387c761673de	form_1.form
camundaForm:2:eaa9ce05-48d7-11ef-b7aa-387c761673de	1	camundaForm	2	eaa97fe3-48d7-11ef-b7aa-387c761673de	form_1.form

It looks like the version has been incremented correctly. What is different about your setup? 🤔

Also, I never saw the error you mentioned.

org.camunda.bpm.engine.ProcessEngineException: ENGINE-03109 Cannot resolve a unique Camunda Form definition for key 'camundaForm' because it exists for multiple tenants.

This error suggests that the form definition might be deployed for multiple tenants. Could that be the case for you? The error message is only produced in this one place. There is a check before if there are zero or one results for a form definition with a given key. In case there are more than one, the error is thrown. The check uses a different SQL statement than the one you changed in your PR. (selectLatestCamundaDefinitionByKeyAndTenantId vs selectLatestCamundaFormDefinitionByKey), so I doubt your PR will solve the error message you are seeing.

Let's try to find out what is different in your setup and if the two problems (error message and form version) are connected. Could you share more information about your setup?

piyushaswani55 commented 1 month ago

Hi @mboskamp This happens only for the forms which are associated with a TENANT_ID, can you please deploy your camundaForm in a TENANT. You can also notice the difference between the queries selectLatestCamundaDefinitionByKeyAndTenantId vs selectLatestCamundaFormDefinitionByKey.

If you focus on the very next method in the file that produces the error message, you would see that camundaFormDefinitionKey is being passed for definitionKey but if you compare the queries then incorrect key is being referenced for selectLatestCamundaDefinitionByKeyAndTenantId while selectLatestCamundaFormDefinitionByKey is referencing to correct key.

mboskamp commented 1 month ago

Hi @piyushaswani55, Thanks for clarifying and confirming my suspicion.

I tried the steps from my previous answer again, this time using a tenant ID. I was able to reproduce the behavior of not incrementing the version number. I can also confirm that your fix works. However, like the last time, I did not see the error message you mentioned (ENGINE-03109). This has to be related to something else on your side.

As I explained earlier, the engine only throws this error in one place which is precisely the method that is not used for finding the latest form definition for a tenant (it is used when not using tenants). So, this PR will not likely fix this error message for you.

I can confirm that the behavior you found is a bug. I will review the PR now. Let's discuss any comments I have over there.

Cheers! Miklas

piyushaswani55 commented 1 month ago

Thanks @mboskamp And yes that error is not re-producible this way, only way to validate the bug is that the form version isn't increasing. That error is indeed related to something else at our end. We have a wrapper API for getTask which fetches the form data and at that time internally we are getting that error due to this bug.

mboskamp commented 1 month ago

Thanks again for fixing this bug. I will merge the PR soon. It will be available with the 7.22.0 release in October. Do you have access to the enterprise releases? I can backport the fix to the supported enterprise versions and it will be available as patch there. Which version do you use?

piyushaswani55 commented 1 month ago

Hi @mboskamp I see the PR is merged. Thank you for your prompt replies and actions. We are not on any enterprise version, and using community edition and are on 7.19.0 version, so I guess, we will have to wait till the version 7.22.0 is officially released and then upgrade to the latest version.

mboskamp commented 1 month ago

@piyushaswani55, the next alpha release will be 7.22.0-alpha4 and go live on Tuesday, 13th of August. You can try out the fix with that version. Our alphas undergo the same tests as the minor release, so it might be worth a try :)

mboskamp commented 1 month ago

Backports:

piyushaswani55 commented 1 month ago

Thanks @mboskamp 😄 This would be really helpful. I would wait for the alpha release and give it a try once available.

camunda / camunda-bpm-platform