HttpExporter not working resulting in 404 page not found

[AKumar9841]

Describe the bug When trying to run the Otel java agent by specifying the below args & config.yaml file ,getting 404 page not found & the below error. [OkHttp http://localhost:4318/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 404. Error message: Unable to parse response body, HTTP status message: Not Found

Steps to reproduce

1. Provide the below system property for open telemetry agent -Dotel.exporter.otlp.endpoint=http://localhost:4318/v1/metrics
2. Configure the config.yaml file for otel-collector. I am using v0.106.1 currently.

   
   receivers:
   otlp:
   protocols:
     http:

processors: batch:

exporters: prometheus: endpoint: "localhost:9090"

service: pipelines: metrics: receivers: [otlp] processors: [batch] exporters: [prometheus]

**What did you expect to see?**
Expected to see httpExporter being able to send the telemetry data to the collector which is listening to port 4318.

**What did you see instead?**
`[OkHttp http://localhost:4318/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 404. Error message: Unable to parse response body, HTTP status message: Not Found`

**What version did you use?**
0.106.1

**What config did you use?**

receivers: otlp: protocols: http:

processors: batch:

exporters: prometheus: endpoint: "localhost:9090"

service: pipelines: metrics: receivers: [otlp] processors: [batch] exporters: [prometheus]

**Environment**
Windows 10

**Additional context**

Even tried using curl command like - `curl http://localhost:4318` , this too was returning 404 page not found.

Please let me know if there are any solutions to the above problem.

[codeboten]

Based on your configuration I would expect the 404 when making a request to http://localhost:4318, but I would expect a 405 when making a GET request to http://localhost:4318/v1/metrics, can you confirm whether you're seeing this as well?

$ curl localhost:4318/v1/metrics
405 method not allowed, supported: [POST]

[AKumar9841]

Hi @codeboten ,

When making a get request to [http://localhost:4318/v1/metrics] , i am getting the below .

curl localhost:4318/v1/metrics
405 method not allowed, supported: [POST]

Could you please suggest how to solve the same ??

[svudya-thy]

Hi, Any solution to this issue? I am facing the same

[AKumar9841]

Hi @svudya-thy ,

What is your collector config could you please share the same ? I have a solution which worked in my case for that I would need to see your collector-config.yaml

[svudya-thy]

Hi @svudya-thy ,

What is your collector config could you please share the same ? I have a solution which worked in my case for that I would need to see your collector-config.yaml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
processors:
  batch:

exporters:
  otlp:
    endpoint: otelcol:4317
  prometheus:
    endpoint: 0.0.0.0:8889
    namespace: otel
  logging:
    loglevel: debug
  otlphttp:
    endpoint: http://loki:3100/otlp

extensions:
  # Default port for health_check extension is 13133
  health_check:
    endpoint: 0.0.0.0:13133
  # Default port for pprof extension is 1888
  pprof:
    endpoint: 0.0.0.0:1888
  # Default port for zpages extension is 55679
  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]

here is my collector config.yml @AKumar9841

[codeboten]

Hi @codeboten ,

When making a get request to [http://localhost:4318/v1/metrics] , i am getting the below .

curl localhost:4318/v1/metrics
405 method not allowed, supported: [POST]

Could you please suggest how to solve the same ??

Right, so the 405 just means it's expecting a request with the POST method. curl lets you specify the HTTP method with -X POST, once you set the method you'll also need to pass in a content type and some data. For example the following request results in a partial success:

$ curl -H 'content-type: application/json' -d '{}' -X POST localhost:4318/v1/metrics
{"partialSuccess":{}}%

I'd recommend looking at tools other than curl to generate telemetry to test your collector though. Something like otel-cli or telemetrygen might be helpful if you're just testing out your setup.

[AKumar9841]

@svudya-thy , what is the type of instrumentation you are using is it Auto via an OtelAgent or Manual Instrumentation via OtelSDK ??

[AKumar9841]

Hi @codeboten , When making a get request to [http://localhost:4318/v1/metrics] , i am getting the below .

curl localhost:4318/v1/metrics
405 method not allowed, supported: [POST]

Could you please suggest how to solve the same ??

Right, so the 405 just means it's expecting a request with the POST method. curl lets you specify the HTTP method with -X POST, once you set the method you'll also need to pass in a content type and some data. For example the following request results in a partial success:

$ curl -H 'content-type: application/json' -d '{}' -X POST localhost:4318/v1/metrics
{"partialSuccess":{}}%

I'd recommend looking at tools other than curl to generate telemetry to test your collector though. Something like otel-cli or telemetrygen might be helpful if you're just testing out your setup.

Hi @codeboten , there was issue with the collector configuration in my case which was fixed by changing the collector receiving port in the yaml file.

Thanks for your help and assistance .

[svudya-thy]

@svudya-thy , what is the type of instrumentation you are using is it Auto via an OtelAgent or Manual Instrumentation via OtelSDK ??

It is Otel agent Java agent to be accurate

[AKumar9841]

@svudya-thy

• What is the error message you are getting ?
• Are you able to get metrics on prometheus ?

[svudya-thy]

@svudya-thy

• What is the error message you are getting ?
• Are you able to get metrics on prometheus ?

It's exactly the same message you got. I am not getting metrics on Prometheus

[AKumar9841]

Got it .

--> Otel agent configuration (i.e. VM args)

-javaagent:\"path\opentelemetry-javaagent.jar\" -Dotel.metrics.exporter=otlp -Dotel.exporter.otlp.metrics.endpoint=http://localhost:4319/v1/metrics -Dotel.service.name=ABCD

--> Prometheus.yml

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.

  - job_name: "collector"
    static_configs:
      - targets: ['collector:5575'] // this should be the collector endpoint

--> collector.yml

receivers:

  otlp:

    protocols:

       http: 

           endpoint: 0.0.0.0:4319

processors:

  batch:    

exporters:

  prometheus:

    endpoint: collector:5575

  logging:

    loglevel: debug    

service:

  pipelines:

    metrics:

      receivers: [otlp]

      processors: [batch]

      exporters: [prometheus]

This should do it.

[AKumar9841]

Hi @svudya-thy ,

Did it work ??