DataDog / datadog-ci

Use Datadog from your CI.
https://datadoghq.com
Apache License 2.0
129 stars 55 forks source link

`datadog-ci junit upload` linked job is not helpful with GitHub #1497

Open eps1lon opened 1 week ago

eps1lon commented 1 week ago

Bug description

@datadog/datadog-ci@2.45.1 junit upload associates a job URL with the test that is not helpful in our case.

The linked job URL will link to the generic "Checks" page of the merge commit in CI (e.g. https://github.com/vercel/next.js/commit/3dd3d19ba63c7ca790f3bf39e0e15152b597547c/checks)

Setting DD_CI_JOB_URL to something we find more useful does not work. In https://github.com/vercel/next.js/pull/73180, we still get the default job name and job url e.g. https://github.com/vercel/next.js/commit/12adaf1a0b0cb362fedc65168c3a15828af2d4e8/checks instead of https://github.com/vercel/next.js/actions/runs/12015965067/attempts/1

Describe what you expected

Linking to the actual job e.g. https://github.com/vercel/next.js/actions/runs/12015964713/job/33495310395.

Since GitHub doesn't expose the job id but only workflow id, the link to the workflow is the next best thing e.g. DD_CI_JOB_URL: https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}

Steps to reproduce the issue

test.yml

steps: 
- name: Upload test report to datadog
  if: ${{ inputs.afterBuild && always() }}
  env:
    DD_ENV: 'ci'
    DD_CI_JOB_NAME: ${{ inputs.stepName }}
    # Better than Datadog's default (e.g. https://github.com/vercel/next.js/commit/3dd3d19ba63c7ca790f3bf39e0e15152b597547c/checks)
    # Job id is not provided by GitHub. Using workflow url as a fallback.
    # We could derived the job id from GH API response but this is just more network slowness.
    DD_CI_JOB_URL: https://github.com/${{  github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}
    run: |
      echo "DD_CI_JOB_NAME: ${{ env.DD_CI_JOB_NAME }}"
      echo "DD_CI_JOB_URL: ${{ env.DD_CI_JOB_URL }}"
      if [ -d ./test/test-junit-report ]; then
        # Add a `test.type` tag to distinguish between turbopack and next.js runs
        npx @datadog/datadog-ci@2.45.1 junit upload --service nextjs --tags test.type:nextjs ./test/test-junit-report
      fi

Additional context

No response

Command

None

juan-fernandez commented 6 days ago

hey @eps1lon , thanks for the report!

I think what you're looking for is the pipeline URL: https://github.com/DataDog/datadog-ci/blob/dcd26a63be14bfdc2eec5ea2ac0dd69277e94373/src/helpers/ci.ts#L235

This URL already has the shape you're expecting, I think:

https://github.com/${{  github.repository }}/actions/runs/${{ github.run_id }}/attempts/${{ github.run_attempt }}

Since GitHub doesn't expose the job id but only workflow id

You're right and that's why we can't offer a deeper link.

Would using pipeline URL instead of job URL work for you? Or did I misunderstand your issue?

eps1lon commented 6 days ago

I'm looking for the Job URL e.g. https://github.com/vercel/next.js/actions/runs/12000501567/job/33449700959. If we'd be able to configure this, we'd have the choice of actually querying the workflow ID from GH Apis. Since it's not configurable, we don't have that choice.

I'll ignore the Job URL until then in favor of the Pipeline URL.

juan-fernandez commented 5 days ago

I see, so you'd like to be able to manually set ci.* tags via DD_CI_* env vars, right?