[Bug]: Remote jaeger endpoint works but local jaeger endpoint errors

Related Problems?

I could not find the bug issue template and so I had to pick this. What is happining is that I have a docker compose that spins up a jaeger and a rust application that I want to output to the jaeger.

services:
  parkinsons_pulse_service:
    image: "pp_service:jaeger"
    restart: unless-stopped
    ports:
      - '4444:4444'
    depends_on:
      - jaeger

  jaeger:
    image: jaegertracing/all-in-one:latest
    restart: unless-stopped
    environment:
      - COLLECTOR_OTLP_ENABLED=true
    ports:
      - '16686:16686'
      - '4317:4317'

tracing = { version = "0.1" }
tracing-subscriber = { version = "0.3" }
tracing-journald = { version = "0.3" }
tracing-opentelemetry = "0.27.0"
opentelemetry = { version = "0.26", features = ["trace", "metrics"] }
opentelemetry-stdout = { version = "0.26", features = ["trace", "metrics"] }
opentelemetry_sdk = { version = "0.26.0", features = ["rt-tokio", "trace"] }
opentelemetry-otlp = { version = "0.26.0", features = ["grpc-tonic", "metrics"] }
opentelemetry-semantic-conventions = "0.26.0"


let mut opentelemetry_layers = Vec::new();
let optl_tracer = opentelemetry_otlp::new_pipeline()
            .tracing()
            .with_trace_config(
                Config::default().with_resource(Resource::new(vec![KeyValue::new(
                    SERVICE_NAME,
                    "parkinsons_pulse_service",
                )])),
            )
            .with_exporter(
                opentelemetry_otlp::new_exporter()
                    .tonic()
                    .with_endpoint("http://localhost:4317"),
            )
            .with_batch_config(BatchConfig::default())
            .install_batch(runtime::Tokio)
            .expect("Failed to initialize tracer provider.");

global::set_tracer_provider(optl_tracer.clone()); // I have tryed commenting this out and changes nothing
let opentelemetry_layer = OpenTelemetryLayer::new(optl_tracer.tracer("tracing-otel-subscriber"));
opentelemetry_layers.push(opentelemetry_layer);

tracing_subscriber::registry()
        .with(opentelemetry_layers)
        .try_init()
        .expect("Could not init tracing registry");

When I run the docker compose locally I get the following error logs

parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603666Z TRACE tower::buffer::service: sending request to buffer worker
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603686Z TRACE tower::buffer::worker: worker polling for next message
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603690Z TRACE tower::buffer::worker: processing new request
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603693Z TRACE tower::buffer::worker: resumed=false worker received request; waiting for service readiness
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603698Z TRACE tonic::transport::channel::service::reconnect: poll_ready; idle
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603704Z TRACE tonic::transport::channel::service::reconnect: poll_ready; connecting
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603711Z TRACE hyper_util::client::legacy::connect::http: Http::connect; scheme=Some("http"), host=Some("localhost"), port=Some(Port(4317))
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603772Z TRACE tonic::transport::channel::service::reconnect: poll_ready; not ready
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603780Z TRACE tower::buffer::worker: service.ready=false delay
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.603824Z DEBUG hyper_util::client::legacy::connect::dns: resolving host="localhost"
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604144Z TRACE tower::buffer::worker: worker polling for next message
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604153Z TRACE tower::buffer::worker: resuming buffered request
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604157Z TRACE tower::buffer::worker: resumed=true worker received request; waiting for service readiness
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604162Z TRACE tonic::transport::channel::service::reconnect: poll_ready; connecting
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604171Z DEBUG hyper_util::client::legacy::connect::http: connecting to [::1]:4317
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604251Z TRACE tonic::transport::channel::service::reconnect: poll_ready; not ready
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604256Z TRACE tower::buffer::worker: service.ready=false delay
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604262Z TRACE tower::buffer::worker: worker polling for next message
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604266Z TRACE tower::buffer::worker: resuming buffered request
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604271Z TRACE tower::buffer::worker: resumed=true worker received request; waiting for service readiness
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604276Z TRACE tonic::transport::channel::service::reconnect: poll_ready; connecting
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604300Z TRACE hyper_util::client::legacy::connect::http: connect error for [::1]:4317: ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604315Z DEBUG hyper_util::client::legacy::connect::http: connecting to 127.0.0.1:4317
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604396Z TRACE tonic::transport::channel::service::reconnect: poll_ready; not ready
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604402Z TRACE tower::buffer::worker: service.ready=false delay
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604408Z TRACE tower::buffer::worker: worker polling for next message
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604412Z TRACE tower::buffer::worker: resuming buffered request
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604414Z TRACE tower::buffer::worker: resumed=true worker received request; waiting for service readiness
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604418Z TRACE tonic::transport::channel::service::reconnect: poll_ready; connecting
parkinsons_pulse_service-1  | 2024-10-14T09:14:20.604432Z TRACE hyper_util::client::legacy::connect::http: connect error for 127.0.0.1:4317: ConnectError("tcp connect error", Os { code: 111, kind: ConnectionRefused, message: "Connection refused" })

However if I remove the Jaeger from running on my local machine ( the docker compose ) and move it to a remote server (raspberry pi sitting next to the computer) and run the docker compose without the Jaeger service on my local machine, it works as intended and I can view all traces from the Jaeger.

Describe the solution you'd like:

Having both contexts work, or details on what is happening.

Considered Alternatives

When I run the example on jaeger it works however I have not tryed to put the example in a docker file and run it that way.

Additional Context

Running NixOs and config is https://github.com/Jack17432/.dotfiles/tree/main/nixos running flake #desktop

open-telemetry / opentelemetry-rust