SigNoz / signoz

SigNoz is an open-source observability platform native to OpenTelemetry with logs, traces and metrics in a single application. An open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool
https://signoz.io
Other
19.17k stars 1.26k forks source link

signoz integration fails when using signoz backend domain name. #996

Open prity-k opened 2 years ago

prity-k commented 2 years ago

Application data is not sent to signoz when OTEL_EXPORTER_OTLP_ENDPOINT is set to signoz backend domain name but it works with the IP. Is there a way to make signoz integration work with domain name?

welcome[bot] commented 2 years ago

Thanks for opening this issue. A team member should give feedback soon. In the meantime, feel free to check out the contributing guidelines.

pranay01 commented 2 years ago

@prity-k Thanks for creating the issue. Quick questions:

  1. Which language/framework is your application written in?
  2. What's the use case for using domain name and not IP? Would be great if you can share in some more detail
  3. Also, can you share the nginx/apache2 configuration that you used
prity-k commented 2 years ago
  1. Python/FastAPI
  2. Signoz is deployed in K8s env and I have configured domain name for the signoz frontend to access dashboards and in similar fashion a domain name has been assigned to signoz backend as well.
  3. `upstream qa--signoz-backend {
    server 10.1.1.230:4317;
    }
    server {
    listen 80;
    server_name signoz-backend.test.com;
    location / {
        proxy_pass http://qa--signoz-backend;
    }
    access_log  /datadrive/logs/nginx/qa--signoz-backend-access.log;
    error_log   /datadrive/logs/nginx/qa--signoz-backend-error.log;
    }`
prashant-shahi commented 2 years ago

proxy_pass http://qa--signoz-backend;

Since 4317 is a gRPC port, we should use grpc_pass instead of proxy_pass.

grpc_pass grpc://qa--signoz-backend;

listen 80;

Since gRPC uses HTTP/2, we should add http2 at the end of the listen command.

listen 80 http2;

@prity-k Do let us know if this works for you.

prity-k commented 2 years ago

@prashant-shahi Unfortunately, above solution did not work. Right now I have following configuration: 1 nginx config:

upstream qa--signoz-backend {
    server 10.1.1.230:4317;
}
server {
    listen 80 http2;
    server_name signoz-backend.test.com;
    location / {
        grpc_pass grpc://qa--signoz-backend;
    }
    access_log  /datadrive/logs/nginx/qa--signoz-backend-access.log;
    error_log   /datadrive/logs/nginx/qa--signoz-backend-error.log;
}
  1. FastAPI deployment.yaml helm file

    
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: {{ .Values.api.name }}
    labels:
    app: {{ .Values.api.name }}
    namespace: {{ .Values.api.namespace }}
    spec:
    replicas: {{ .Values.api.replicaCount }}
    selector:
    matchLabels:
      app: {{ .Values.api.name }}
    template:
    metadata:
      labels:
        app: {{ .Values.api.label }}
    spec:
      containers:
        - name: {{ .Values.api.name }}
          image: {{ .Values.api.image }}
          env:
          - name: OTEL_RESOURCE_ATTRIBUTES
            value: {{ .Values.signoz.appTag }}
          - name: OTEL_EXPORTER_OTLP_ENDPOINT
            value: {{ .Values.signoz.signozBackend }}
          imagePullPolicy: {{ .Values.api.pullPolicy }}
          ports:
            - name: http
              containerPort: {{ .Values.api.containerPort }}
              protocol: TCP
    
      imagePullSecrets:
        - name: {{ .Values.api.imagePullSecrets }}

3. values.yaml

signoz: signozBackend: http://signoz-backend.test.com appTag: service.name=my-api