search

DataDog / dd-trace-py

Datadog Python APM Client
https://ddtrace.readthedocs.io/
Other
546 stars 411 forks source link

Git Metadata errors from CI Visibility despite explicit metdata configuration and disabling metadata #10983

Open jasondamour opened 1 week ago

jasondamour commented 1 week ago

Hello,

Our tests run in CI without a .git/ directory intentionally. We're configuring CI Visibility git metadata with DD_GIT_REPOSITORY_URL, DD_GIT_BRANCH, and DD_GIT_COMMIT_SHA. However, the ddtrace agent is still printing lots of errors about not finding git. Our configuration is below:

ddtrace: 2.11.6
# env
DD_CIVISIBILITY_AGENTLESS_ENABLED=true
DD_TRACE_GIT_METADATA_ENABLED=false
DD_ENV=CI
DD_SERVICE=xxx
DD_API_KEY=xxx
DD_APP_KEY=xxx
DD_GIT_REPOSITORY_URL=xxx
DD_GIT_BRANCH=xxx
DD_GIT_COMMIT_SHA=xxx

And tests are executed with pytest ..... --ddtrace:

[Datadog CI Visibility] ERROR    ddtrace.ext.git:git.py:146 Error setting safe directory
[Datadog CI Visibility] ERROR    ddtrace.ext.git:git.py:339 Error extracting git metadata: fatal: No remote configured to list refs from.
Process Process-1:
Traceback (most recent call last):
  File "/Users/jasondamour/.pyenv/versions/3.10.6/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/Users/jasondamour/.pyenv/versions/3.10.6/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/jasondamour/.cache/pants/named_caches/pex_root/venvs/s/15dd8c43/venv/lib/python3.10/site-packages/ddtrace/internal/ci_visibility/git_client.py", line 162, in _run_protocol
    latest_commits = cls._get_latest_commits(cwd=cwd)
  File "/Users/jasondamour/.cache/pants/named_caches/pex_root/venvs/s/15dd8c43/venv/lib/python3.10/site-packages/ddtrace/internal/ci_visibility/git_client.py", line 238, in _get_latest_commits
    raise ValueError(stderr)
ValueError: fatal: not a git repository (or any of the parent directories): .git
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute

Asks:

  1. These errors should not occur since the metadata is explicitly provided
  2. CI Visibility should expose a variable for explicitly disabling the git client (or respect the existing DD_TRACE_GIT_METADATA_ENABLED variable) since git metadata is optional anyways
brettdh commented 1 day ago

I've run into this as well. Digging into the code a bit, it turns out that this code path isn't trying to gather any metadata that can be provided (today) via environment variables. Instead, it's gathering a list of up to 1000 commits in the past month (git command), which it then uploads to Datadog. Presumably this is needed for error attribution or other features related to change detection.

I'm working on a PR that'll be either: