Failing to backport because of merge conflicts should mark the github action step in error

[sarod]

Hi,

I followed the setup described in the README .

When the backport fails because of merge conflicts the action logs the following JSON object:

{
  "status": "success",
  "commits": [
    {
      "author": {
        "name": "Sebastien Arod",
        "email": "sebastien.arod@xxx.com"
      },
      "sourceCommit": {          
        "committedDate": "2022-05-20T15:08:34Z",
        "message": "xxx.",
        "sha": "dbdcace0fde6e50c46c4b929b3500e8b2aae8cc4"
      },
      "sourcePullRequest": {
        "number": 784,
        "url": "https://github.com/xxx/xxx/pull/784",
        "mergeCommit": {
          "message": "xxx.",
          "sha": "dbdcace0fde6e50c46c4b929b3500e8b2aae8cc4"
        }
      },
      "sourceBranch": "master",
      "expectedTargetPullRequests": [
        {
          "branch": "V5_3",
          "state": "NOT_CREATED"
        }
      ]
    }
  ],
  "results": [
    {
      "targetBranch": "V5_3",
      "status": "handled-error",
      "error": {
        "name": "BackportError",
        "errorContext": {
          "code": "merge-conflict-exception",
          "commitsWithoutBackports": [],
          "conflictingFiles": [
            "xxx.xxx"
          ]
        }
      }
    }
  ]
}

The results property part clearly indicate that there was some merge-conflict-exception but the global status is success and the step is considered successful.

Is this the expected behavior? I would have expected a failed backport to fail the step.

[sorenlouv]

Is this the expected behavior? I would have expected a failed backport to fail the step.

This is somewhat expected behaviour. It's not set in stone but my intention is to only mark operational failures as "failed". This way an ops person can verify whether the github action is performing as expected by monitoring the failure rate.

If merge conflicts are marked as failured it becomes very hard to spot if there are operational failures (eg. backport tool crashing) vs soft failures (merge conflicts).

I'd love it if it was possible to mark a job with other states than simply "success" or "failure" but it doesn't look like it.

[sarod]

That makes sense. Thanks!

[sorenlouv]

Closing for now. Feel free to comment if this comes up again