search

open-telemetry / opentelemetry-demo

This repository contains the OpenTelemetry Astronomy Shop, a microservice-based distributed system intended to illustrate the implementation of OpenTelemetry in a near real-world environment.
https://opentelemetry.io/docs/demo/
Apache License 2.0
1.53k stars 869 forks source link

Grafana doesn't work without an Internet connection #1627

Open flands opened 1 week ago

flands commented 1 week ago

Bug Report

Which version of the demo you are using? 1.10.0

Symptom

If you attempt to start the demo application without an Internet connection (containers already pulled down), then Grafana is not access and restarts every minute

What is the expected behavior?

If you have the demo environment containers locally, then the demo environment should work without an Internet connection.

What is the actual behavior?

The demo environment requires an Internet connection for Grafana to work even if container are already pulled down.

Reproduce

Pull version 1.10.0 and run make start-minimal with an Internet connection. Once running, run make stop. Disable your Internet. Run make start-minmal. Once running, attempt to access Grafana.

Additional Context

If you look at the logs for Grafana without an Internet connection you will see: Error: ✗ Get "https://grafana.com/api/plugins/grafana-opensearch-datasource/versions": dial tcp: lookup grafana.com on 127.0.0.11:53: no such host

I see several potential issues. The root cause appears to be because src/grafana/provisioning/datasources/opensearch.yaml has access: proxy set. If you remove this, the issue is resolved. I can make this change if everyone is aligned.

Beyond this, it appears src/grafana/grafana.ini has check_for_updates = true and check_for_plugin_updates = true. I'm not sure whether this will result in Grafana being unavailable, but also seems unnecessary for the demo environments. I can make this change if everyone is aligned.

julianocosta89 commented 6 days ago

I'm good with the suggestions. I actually faced the same during my flight this week 😅 I wanted to run the demo to test the feature flags issue you reported and I saw that Grafana wasn't starting.

flands commented 6 days ago

OK, I will get a PR up!

flands commented 2 days ago

Turns out this is a bit more complicated - see https://github.com/open-telemetry/opentelemetry-demo/pull/1633#issuecomment-2198552666.