Beta-4.2-SNAPSHOT fails to reopen closed workitem in ADO

[Ri7Sh]

BugTrackerUtility fails to reopen closed workitems in ADO with the Beta4.2-SNAPSHOT while with TFS in older version, this feature was working as expected. Following are the logs:

2021-04-20 18:15:41,802 [main] ERROR com.fortify.bugtracker.common.tgt.processor.AbstractTargetProcessorUpdateIssues - [ADO] Error updating issue https://{ADOBaseurl}/{ADOProject}/_workitems/edit/{workitemid}
java.lang.RuntimeException: Error accessing remote system https://{BaseUrl}: Bad Request
    at com.fortify.util.rest.connection.AbstractRestConnection.getUnsuccesfulResponseException(AbstractRestConnection.java:358) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.util.rest.connection.AbstractRestConnection.checkResponseAndGetOutput(AbstractRestConnection.java:322) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.util.rest.connection.AbstractRestConnection.executeRequest(AbstractRestConnection.java:224) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.util.rest.connection.AbstractRestConnection.executeRequestWithFinalizedWebTarget(AbstractRestConnection.java:186) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.util.rest.connection.AbstractRestConnection.executeRequest(AbstractRestConnection.java:167) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.tgt.ado.connection.ADORestConnection.updateIssueData(ADORestConnection.java:86) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.tgt.ado.processor.ADOTargetProcessorUpdateIssuesWithTransitions.transition(ADOTargetProcessorUpdateIssuesWithTransitions.java:91) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.tgt.processor.AbstractTargetProcessorUpdateIssuesWithTransitions.transition(AbstractTargetProcessorUpdateIssuesWithTransitions.java:110) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.tgt.processor.AbstractTargetProcessorUpdateIssuesWithTransitions.openIssueIfClosed(AbstractTargetProcessorUpdateIssuesWithTransitions.java:78) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.tgt.processor.AbstractTargetProcessorUpdateIssues.openIssueIfNecessary(AbstractTargetProcessorUpdateIssues.java:242) ~[FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.tgt.processor.AbstractTargetProcessorUpdateIssues.processMap(AbstractTargetProcessorUpdateIssues.java:114) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessorBuildObjectMapFromGroupedObjects.processGroup(AbstractProcessorBuildObjectMapFromGroupedObjects.java:71) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessorGroupByExpressions.postProcess(AbstractProcessorGroupByExpressions.java:168) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor._process(AbstractProcessor.java:102) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor.process(AbstractProcessor.java:82) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.src.processor.AbstractSourceProcessorRetrieveVulnerabilities.process(AbstractSourceProcessorRetrieveVulnerabilities.java:78) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor._process(AbstractProcessor.java:101) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor.process(AbstractProcessor.java:82) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.bugtracker.common.src.processor.AbstractSourceVulnerabilityProcessor.process(AbstractSourceVulnerabilityProcessor.java:90) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractCompositeProcessor.processForAll(AbstractCompositeProcessor.java:112) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractCompositeProcessor.process(AbstractCompositeProcessor.java:85) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor._process(AbstractProcessor.java:101) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.processor.AbstractProcessor.process(AbstractProcessor.java:82) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.AbstractProcessRunner.process(AbstractProcessRunner.java:72) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.AbstractProcessRunner.run(AbstractProcessRunner.java:55) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.RunProcessRunnerFromSpringConfig.run(RunProcessRunnerFromSpringConfig.java:85) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.RunProcessRunnerFromCLI.run(RunProcessRunnerFromCLI.java:166) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
    at com.fortify.processrunner.RunProcessRunnerFromCLI.main(RunProcessRunnerFromCLI.java:360) [FortifyBugTrackerUtility-4.2-SNAPSHOT.jar:?]
Caused by: java.lang.Exception: Error accessing remote system https://msit-test1.visualstudio.com: Bad Request, response contents: 
{"$id":"1","customProperties":{"FieldReferenceName":null,"FieldStatusFlags":"none","ErrorMessage":"TF401320: Rule Error for field Resolved Reason. Error code: HasValues, LimitedToValues, SetByRule, InvalidNotEmpty.","FieldStatusCode":0,"RuleValidationErrors":[{"fieldReferenceName":"Microsoft.VSTS.Common.ResolvedReason","fieldStatusFlags":"hasValues, limitedToValues, setByRule, invalidNotEmpty","errorMessage":"TF401320: Rule Error for field Resolved Reason. Error code: HasValues, LimitedToValues, SetByRule, InvalidNotEmpty.","fieldStatusCode":1048716,"ruleValidationErrors":null}]},"innerException":null,"message":"TF401320: Rule Error for field Resolved Reason. Error code: HasValues, LimitedToValues, SetByRule, InvalidNotEmpty.","typeName":"Microsoft.TeamFoundation.WorkItemTracking.Server.RuleValidationException, Microsoft.TeamFoundation.WorkItemTracking.Server, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a","typeKey":"RuleValidationException","errorCode":0,"eventId":3200}
    ... 28 more

[Ri7Sh]

https://github.com/fortify-ps/FortifyBugTrackerUtility/blob/2bd4e6d7510e13adae6c012d88553a1ef5e252ca/bugtracker-target-ado/src/main/java/com/fortify/bugtracker/tgt/ado/connection/ADORestConnection.java#L86

This issue is due to the request body created at the above code

entity=[{
  "op":"add",
  "path":"/fields/System.History",
  "value":"Bug re-opened as the issue was detected again in the most recent Fortify scan"
}, {
 "op":"add",
 "path":"/fields/System.State",
 "value":"New"
}]

One of the solution to remediate this can be for this PATCH, Microsoft.VSTS.Common.ResolvedReason field should also be updated alongside and formed entity should be

entity=[{
  "op":"add",
  "path":"/fields/System.History",
  "value":"Bug re-opened as the issue was detected again in the most recent Fortify scan"
}, {
 "op":"add",
 "path":"/fields/System.State",
 "value":"New"
},{
 "op": "add",
  "path": "/fields/Microsoft.VSTS.Common.ResolvedReason",
  "value": ""
}]

[rsenden]

@Ri7Sh I cannot reproduce this error; Azure DevOps seems to automatically reset the Resolved Reason field when changing the state from Resolved to Active, see screenshot below.

From the configuration file you shared in #54, it seems like you have customized the transition configuration, so maybe this error only occurs on the transitions that you have configured. I guess all of this is also highly dependent on the ADO process and work item type that you are using, and the corresponding work item rules in ADO.

If this was working before, then I guess some of the rules on TFS/ADO have changed; there have been no significant changes in this area in the bug tracker utility. I don't want to simply set the ResolvedReason field to an empty string, as I don't know the impact of this when using other ADO processes or work item types.

Please work with your Azure DevOps team to see whether any rules could be updated to prevent this from happening. Potentially I could add support for configuring additional field values during transitions, such that users can add such operations by themselves for a specific process or work item type, without potentially affecting other ADO processes or work item types.

[Ri7Sh]

@rsenden I am facing this issue for WorkItem type = Bug, and any transition from Resolved/Closed Bug State to Active/New Bug State. But, as you mentioned, it might be an error specific to our ADO instance.

[rsenden]

@Ri7Sh Can you please check with your ADO team whether this might be specific to your ADO instance? In that case it may be easier to have them change the rules to allow these transitions without having to set the ResolvedReason field to an empty string, rather than waiting for me to implement a new feature for customizable fields contents on transitions. Please close this issue if you are planning to resolve this with the help of your ADO team.

[Ri7Sh]

Sure, I will check with the ADO team.