Telemetry not being generated and exported as expected.

Describe the bug

Telemetry not being generated and exported as expected. I am trying to use the otel environment variables to generate data I can feed to otel collector and subsequently to prometheus and Grafana. The issue is that logs aren't generated as expected. As data comes through from other apps and I debugged each component in the architecture I can only conclude it's happening on infisical's side.

To Reproduce

Steps to reproduce the behavior: 1 run the following setup.

version: "3"

services:
  db-migration:
    container_name: infisical-db-migration
    depends_on:
      db:
        condition: service_healthy
    image: infisical/infisical:v0.95.0-postgres
    env_file: .env
    command: npm run migration:latest
    pull_policy: always
    networks:
      - infisical

  backend:
    container_name: infisical-backend
    restart: unless-stopped
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started
      db-migration:
        condition: service_completed_successfully
    image: infisical/infisical:v0.95.0-postgres
    pull_policy: always
    env_file: .env
    ports:
      - "9464:9464"
    environment:
      - NODE_ENV=production
    networks:
      - infisical
      - reverse-proxy
      - telemetry-network

  redis:
    image: redis:alpine3.20
    container_name: infisical-dev-redis
    env_file: .env
    restart: always
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
    networks:
      - infisical
    volumes:
      - redis_data:/data

  db:
    container_name: infisical-db
    image: postgres:14-alpine
    restart: always
    env_file: .env
    volumes:
      - pg_data:/var/lib/postgresql/data
    networks:
      - infisical
    healthcheck:
      test: "pg_isready --username=${POSTGRES_USER} && psql --username=${POSTGRES_USER} --list"
      interval: 5s
      timeout: 10s
      retries: 10

volumes:
  pg_data:
    driver: local
  redis_data:
    driver: local

networks:
  infisical:
  telemetry-network:
    external: true
  reverse-proxy:
    external: true

With the following environment variables:

OTEL_TELEMETRY_COLLECTION_ENABLED=true
OTEL_EXPORT_TYPE=otlp
OTEL_EXPORT_OTLP_ENDPOINT=http://otel_collector:4318

No data flows into Prometheus. If I change the export type to Prometheus and manually curl I get the following result:

$ curl http://localhost:9464/metrics
curl: (56) Recv failure: Connection reset by peer

Expected behavior

Logs and/or traces being created and delivered to open telemetry connector.

Screenshots

If applicable, add screenshots to help explain your problem.

Platform you are having the issue on:

Additional context

For detail's sake, I'm running the otel_collector in a different compose file inside the telemetry-network.

Infisical / infisical