Can't see metrics nor traces in opentelemetry collector

Describe the bug We have an app that exports traces and metrics and that is now instrumented as described in this comment. Previously we used opentelemetry-collector-contrib version 0.38.0 and we were seeing both metrics and traces come through. Since the upgrade to 0.46.0 this has broken. We confirmed that the metrics are getting exported by the app by changing otel.exporter.otlp.endpoint property to something other than localhost:4317 (OkHttpGrpcExporter reacted by throwing errors).

Steps to reproduce We use docker-compose to set-up our collector and apm server. These are the relevant lines from the docker-compose.yml file:

version: "3.7"

services:
  elasticsearch:
    ...
  kibana:
   ...
    depends_on:
      - elasticsearch

  apm-server:
    container_name: apm-server
    image: docker.elastic.co/apm/apm-server:7.14.2
    restart: always
    user: root
    networks:
      - elastic
    ports:
      - 8200:8200
    volumes:
      - ./apm-server.yml:/usr/share/apm-serer/apm-server.yml:ro
    command: ["--strict.perms=false", "-system.hostfs=/hostfs"]
    depends_on:
      - kibana
      - elasticsearch

  otel-collector:
    container_name: otel-collector
    image: otel/opentelemetry-collector-contrib:0.46.0
    command: [ "--config=/etc/otelcol-contrib/config.yaml" ]
    working_dir: "/project"
    volumes:
      - ${PWD}/:/project
      - ./otel-collector-config.yml:/etc/otelcol-contrib/config.yaml
      - ./our-app/opentelemetry-jmx-metrics.jar:/opentelemetry-jmx-metrics.jar
    ports:
      - "4317:4317"
      - "4318:4318"
    depends_on:
      - apm-server
    networks:
      - elastic

  metricbeat:
    ...

  filebeat:
    ...

networks:
  elastic:

volumes:
  ...

On the application side we use the autoconfigure module - both opentelemetry-api and javaagent are versions 1.12.0.

What did you expect to see? With debug level logging enabled ☝️ we would expect to see metrics (gauges, histogram, etc.) from the application appear in followed logs.

What did you see instead? There is plenty of metrics such as queueSize, runtime.jvm.memory.area etc. but nothing application specific.

What version did you use? Version: v0.46.0

What config did you use? This is the referenced in docker-compose.yml otel-collector-config.yml file:

receivers:
  otlp/receiver:
    protocols:
      grpc:
      http:

exporters:
  otlp/elastic:
    endpoint: apm-server:8200
    tls:
      insecure: true
    sending_queue:
      enabled: true
      num_consumers: 15
      queue_size: 7500
  logging:
    loglevel: debug

processors:
  batch:
    timeout: 10s
  memory_limiter:
    check_interval: 1s
    limit_percentage: 70
    spike_limit_percentage: 30

service:
  pipelines:
    traces:
      receivers: [otlp/receiver]
      processors: [memory_limiter,batch]
      exporters: [otlp/elastic]
    metrics:
      receivers: [otlp/receiver]
      processors: [memory_limiter,batch]
      exporters: [otlp/elastic, logging]

Environment Compiler: openjdk 11.0.11 2021-04-20 OS: MacOS Monterey 12.3

Additional context As an example, there was a configuration change about the expected config location (I think the new expected location is /etc/otelcol-contrib/).

open-telemetry / opentelemetry-collector-contrib

Can't see metrics nor traces in opentelemetry collector #8851