datadog agent failing to connect to custom datadog host with SSL errors.

[pdulimitta]

We have a custom datadog host(Example: https://.datadoghq.com/. The docker version of datadog agent that we are running on our local mac for testing/development purpose is failing to connect to datadaog host with SSL/TLS errors.

The error we see in the logs is:

dd-agent        | 2024-02-10 23:38:42 UTC | CORE | ERROR | (comp/forwarder/defaultforwarder/worker.go:187 in process) | Too many errors for endpoint 'https://app.<custom>.datadoghq.com/api/v2/series': retrying later

dd-agent        | 2024-02-10 23:38:45 UTC | CORE | WARN | (pkg/logs/client/http/destination.go:220 in sendAndRetry) | Could not send payload: Post "https://contlcycle-intake.<custom>.datadoghq.com/api/v2/contlcycle": tls: failed to verify certificate: x509: certificate is valid for *.datadoghq.com, datadoghq.com, not contlcycle-intake.<custom>.datadoghq.com

Can someone please suggest what is the best way to resolve this issue.

Sharing my datadog agent from docker-compose.yaml

  datadog:
    image: datadog/agent
    container_name: dd-agent
    restart: always
    environment:
      - DD_API_KEY=<my-api-key>
      - DD_SITE=<my-custom>.datadoghq.com
      - DD_HOSTNAME=localhost
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /proc/:/host/proc/:ro
      - /sys/fs/cgroup/:/host/sys/fs/cgroup:ro

[molnare]

Having the same issue. Installed standalone datadog-agent on an Azure VM, trying to send to custom datadog host .datadoghq.com. Example error:

tls: failed to verify certificate: x509: certificate is valid for *.datadoghq.com, datadoghq.
com, not config.<custom>.datadoghq.com

There are many like this, config.\<custom>.datadoghq.com is not the only one affected.

[sgnn7]

@molnare / @pdulimitta Can you try to run things without including the custom site prefix? I believe that the custom hosts are only relevant for the website access, not the ingestion of data (though I could be wrong).