search

grafana / oncall

Developer-friendly incident response with brilliant Slack integration
GNU Affero General Public License v3.0
3.51k stars 291 forks source link

Unable to connect to grafana oncall from external grafana #1035

Closed midhunkr1shna closed 8 months ago

midhunkr1shna commented 1 year ago

It worked while running Grafana and Grafana OnCall using the Docker-compose file from this repository.But

Error shows up while connecting to oncall from other grafana sources.

" Error -An unknown error occured when trying to install the plugin. Are you sure that your OnCall API URL, http://xxx.xxx.xx:8080, is correct? Refresh your page and try again, or try removing your plugin configuration and reconfiguring."

Matvey-Kuk commented 1 year ago

Hi! Thank you for the report. Could you please provide logs from engine and celery worker?

OlegSupport commented 1 year ago

i have the the same error

midhunkr1shna commented 1 year ago

Thanks for the reply ,

We cannot find any logs related to the above issue in engine or celery worker. But grafana is showing a 404 error

" logger=context traceID=00000000000000000000000000000000 userId=4 orgId=5 uname=user@example.com t=2022-12-22T12:25:48.095309245Z level=info msg="Request Completed" method=POST path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/self-hosted/install status=404 remote_addr=10.10.x.x time_ms=5 duration=5.555156ms size=24 referer=https://grafana.example.com/plugins/grafana-oncall-app traceID=00000000000000000000000000000000"

Curl request from the same grafana machine to engine:8080 is giving a response "OK"

tmpm697 commented 1 year ago

I got same issue. Can't connect oncall with external grafana, details: https://grafana.slack.com/archives/C02LSUUSE2G/p1672041357673719

tmpm697 commented 1 year ago

Use grafana and grafana oncall from docker-compose file also didn't work for me, I got these error from grafana (I tried input localhost:8080, http://localhost:8080 when start plugin):

logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/sync remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:12:35.088290331Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:12:35.088354191Z level=error msg="Request Completed" method=POST path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/sync status=502 remote_addr=172.24.0.1 time_ms=18 duration=18.221173ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/sync remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:12:35.262935849Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:12:35.263009103Z level=error msg="Request Completed" method=POST path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/sync status=502 remote_addr=172.24.0.1 time_ms=11 duration=11.800825ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:13:14.172133305Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:13:14.172188936Z level=error msg="Request Completed" method=GET path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status status=502 remote_addr=172.24.0.1 time_ms=12 duration=12.236542ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=plugindashboards t=2022-12-26T12:13:16.231169097Z level=info msg="Plugin state changed" pluginId=grafana-oncall-app enabled=false
logger=plugindashboards t=2022-12-26T12:13:20.67915993Z level=info msg="Plugin state changed" pluginId=grafana-oncall-app enabled=true
logger=plugindashboards t=2022-12-26T12:13:20.679248456Z level=info msg="Syncing plugin dashboards to DB" pluginId=grafana-oncall-app
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/self-hosted/install remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:13:20.865888946Z level=error msg="Proxy request failed" err="unsupported protocol scheme \"localhost\""
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:13:20.865960958Z level=error msg="Request Completed" method=POST path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/self-hosted/install status=502 remote_addr=172.24.0.1 time_ms=28 duration=28.262888ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:13:33.247855204Z level=info msg="Request Completed" method=GET path=/api/live/ws status=0 remote_addr=172.24.0.1 time_ms=1 duration=1.990658ms size=0 referer= traceID=00000000000000000000000000000000
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:13:35.024210444Z level=error msg="Proxy request failed" err="unsupported protocol scheme \"localhost\""
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:13:35.024277811Z level=error msg="Request Completed" method=GET path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status status=502 remote_addr=172.24.0.1 time_ms=23 duration=23.097416ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=0 uname= t=2022-12-26T12:13:51.504996058Z level=info msg="Request Completed" method=GET path=/ status=302 remote_addr=172.24.0.1 time_ms=1 duration=1.08014ms size=29 referer= traceID=00000000000000000000000000000000
logger=plugindashboards t=2022-12-26T12:14:50.243353303Z level=info msg="Plugin state changed" pluginId=grafana-oncall-app enabled=false
logger=plugindashboards t=2022-12-26T12:14:55.746914762Z level=info msg="Plugin state changed" pluginId=grafana-oncall-app enabled=true
logger=plugindashboards t=2022-12-26T12:14:55.746959153Z level=info msg="Syncing plugin dashboards to DB" pluginId=grafana-oncall-app
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/self-hosted/install remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:14:55.923077152Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:14:55.923151618Z level=error msg="Request Completed" method=POST path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/self-hosted/install status=502 remote_addr=172.24.0.1 time_ms=25 duration=25.255827ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:14:58.190076565Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:14:58.190142115Z level=error msg="Request Completed" method=GET path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status status=502 remote_addr=172.24.0.1 time_ms=23 duration=23.096838ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:15:00.692002945Z level=info msg="Request Completed" method=GET path=/api/live/ws status=0 remote_addr=172.24.0.1 time_ms=1 duration=1.509684ms size=0 referer= traceID=00000000000000000000000000000000
logger=data-proxy-log userId=1 orgId=1 uname=admin app=grafana-oncall-app path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status remote_addr=172.24.0.1 referer=http://localhost:3000/plugins/grafana-oncall-app t=2022-12-26T12:15:02.809682614Z level=error msg="Proxy request failed" err="dial tcp 127.0.0.1:8080: connect: connection refused"
logger=context traceID=00000000000000000000000000000000 userId=1 orgId=1 uname=admin t=2022-12-26T12:15:02.809746433Z level=error msg="Request Completed" method=GET path=/api/plugin-proxy/grafana-oncall-app/api/internal/v1/plugin/status status=502 remote_addr=172.24.0.1 time_ms=20 duration=20.573217ms size=0 referer=http://localhost:3000/plugins/grafana-oncall-app traceID=00000000000000000000000000000000
logger=context traceID=00000000000000000000000000000000 userId=0 orgId=0 uname= t=2022-12-26T12:15:15.540783978Z level=info msg="Request Completed" method=GET path=/ status=302 remote_addr=172.24.0.1 time_ms=0 duration=819.141µs size=29 referer= traceID=00000000000000000000000000000000

and these from celery:

2022-12-26 12:13:17,594 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task restore_heartbeat_tasks (apps.heartbeat.tasks.restore_heartbeat_tasks)
2022-12-26 12:13:17,596 source=engine:celery worker=ForkPoolWorker-2 task_id=91c7bd4c-01c9-459d-ab7a-e376b4e1d8d0 task_name=apps.schedules.tasks.refresh_ical_files.start_refresh_ical_files name=apps.schedules.tasks.refresh_ical_files level=INFO Start refresh ical files
2022-12-26 12:13:17,597 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.heartbeat.tasks.restore_heartbeat_tasks[159a6a46-2396-447a-b95c-8b643b800703] received
2022-12-26 12:13:17,598 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task process_failed_to_invoke_celery_tasks (apps.base.tasks.process_failed_to_invoke_celery_tasks)
2022-12-26 12:13:17,601 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.base.tasks.process_failed_to_invoke_celery_tasks[d1fdbf07-67b7-43b0-8a1e-ebe3240316b3] received
2022-12-26 12:13:17,602 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task check_escalations (apps.alerts.tasks.check_escalation_finished.check_escalation_finished_task)
2022-12-26 12:13:17,604 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.alerts.tasks.check_escalation_finished.check_escalation_finished_task[8d2c57cd-97a4-4481-aad0-02d70050247b] received
2022-12-26 12:13:17,606 source=engine:celery worker=ForkPoolWorker-2 task_id=91c7bd4c-01c9-459d-ab7a-e376b4e1d8d0 task_name=apps.schedules.tasks.refresh_ical_files.start_refresh_ical_files name=celery.app.trace level=INFO Task apps.schedules.tasks.refresh_ical_files.start_refresh_ical_files[91c7bd4c-01c9-459d-ab7a-e376b4e1d8d0] succeeded in 0.010021762005635537s: None
2022-12-26 12:13:17,612 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.slack.tasks.start_update_slack_user_group_for_schedules[d0b8d692-a939-471a-b891-6eef7ddcc84f] received
2022-12-26 12:13:17,616 source=engine:celery worker=ForkPoolWorker-2 task_id=159a6a46-2396-447a-b95c-8b643b800703 task_name=apps.heartbeat.tasks.restore_heartbeat_tasks name=celery.app.trace level=INFO Task apps.heartbeat.tasks.restore_heartbeat_tasks[159a6a46-2396-447a-b95c-8b643b800703] succeeded in 0.006284839000727516s: None
2022-12-26 12:13:17,625 source=engine:celery worker=ForkPoolWorker-2 task_id=d1fdbf07-67b7-43b0-8a1e-ebe3240316b3 task_name=apps.base.tasks.process_failed_to_invoke_celery_tasks name=celery.app.trace level=INFO Task apps.base.tasks.process_failed_to_invoke_celery_tasks[d1fdbf07-67b7-43b0-8a1e-ebe3240316b3] succeeded in 0.0055856629987829365s: None
2022-12-26 12:13:17,635 source=engine:celery worker=ForkPoolWorker-2 task_id=8d2c57cd-97a4-4481-aad0-02d70050247b task_name=apps.alerts.tasks.check_escalation_finished.check_escalation_finished_task name=celery.app.trace level=INFO Task apps.alerts.tasks.check_escalation_finished.check_escalation_finished_task[8d2c57cd-97a4-4481-aad0-02d70050247b] succeeded in 0.007020125995040871s: None
2022-12-26 12:13:47,794 source=engine:celery worker=ForkPoolWorker-2 task_id=d0b8d692-a939-471a-b891-6eef7ddcc84f task_name=apps.slack.tasks.start_update_slack_user_group_for_schedules name=celery.app.trace level=INFO Task apps.slack.tasks.start_update_slack_user_group_for_schedules[d0b8d692-a939-471a-b891-6eef7ddcc84f] succeeded in 0.011868301997310482s: None
2022-12-26 12:14:00,032 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task sync_users_with_cloud (apps.oss_installation.tasks.sync_users_with_cloud)
2022-12-26 12:14:00,037 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[d3e3a81a-73d0-4a14-a432-524093951326] received
2022-12-26 12:14:00,040 source=engine:celery worker=ForkPoolWorker-2 task_id=d3e3a81a-73d0-4a14-a432-524093951326 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Start sync_users_with_cloud
2022-12-26 12:14:00,047 source=engine:celery worker=ForkPoolWorker-2 task_id=d3e3a81a-73d0-4a14-a432-524093951326 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Grafana Cloud is not connected
2022-12-26 12:14:00,048 source=engine:celery worker=ForkPoolWorker-2 task_id=d3e3a81a-73d0-4a14-a432-524093951326 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=celery.app.trace level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[d3e3a81a-73d0-4a14-a432-524093951326] succeeded in 0.007740451997960918s: None
2022-12-26 12:15:00,060 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task start_cleanup_deleted_organizations (apps.grafana_plugin.tasks.sync.start_cleanup_deleted_organizations)
2022-12-26 12:15:00,063 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.grafana_plugin.tasks.sync.start_cleanup_deleted_organizations[321b0508-8a6c-49c1-ba22-e67f881ae1cc] received
2022-12-26 12:15:00,063 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task send_cloud_heartbeat (apps.oss_installation.tasks.send_cloud_heartbeat_task)
2022-12-26 12:15:00,067 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task sync_users_with_cloud (apps.oss_installation.tasks.sync_users_with_cloud)
2022-12-26 12:15:00,067 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.oss_installation.tasks.send_cloud_heartbeat_task[770346a3-8838-4f77-bb8b-1a5e112d6dcf] received
2022-12-26 12:15:00,070 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[51e4075b-8b9d-4903-87dd-ba8fcb6cbd1e] received
2022-12-26 12:15:00,073 source=engine:celery worker=ForkPoolWorker-2 task_id=321b0508-8a6c-49c1-ba22-e67f881ae1cc task_name=apps.grafana_plugin.tasks.sync.start_cleanup_deleted_organizations name=apps.grafana_plugin.tasks.sync level=DEBUG Found 0 deleted organizations not synced recently
2022-12-26 12:15:00,074 source=engine:celery worker=ForkPoolWorker-2 task_id=321b0508-8a6c-49c1-ba22-e67f881ae1cc task_name=apps.grafana_plugin.tasks.sync.start_cleanup_deleted_organizations name=celery.app.trace level=INFO Task apps.grafana_plugin.tasks.sync.start_cleanup_deleted_organizations[321b0508-8a6c-49c1-ba22-e67f881ae1cc] succeeded in 0.008622757995908614s: None
2022-12-26 12:15:00,087 source=engine:celery worker=ForkPoolWorker-2 task_id=770346a3-8838-4f77-bb8b-1a5e112d6dcf task_name=apps.oss_installation.tasks.send_cloud_heartbeat_task name=apps.oss_installation.cloud_heartbeat level=INFO Unable to send cloud heartbeat. Check values for GRAFANA_CLOUD_ONCALL_HEARTBEAT_ENABLED and GRAFANA_CLOUD_ONCALL_TOKEN.
2022-12-26 12:15:00,088 source=engine:celery worker=ForkPoolWorker-2 task_id=770346a3-8838-4f77-bb8b-1a5e112d6dcf task_name=apps.oss_installation.tasks.send_cloud_heartbeat_task name=celery.app.trace level=INFO Task apps.oss_installation.tasks.send_cloud_heartbeat_task[770346a3-8838-4f77-bb8b-1a5e112d6dcf] succeeded in 0.010192585999902803s: None
2022-12-26 12:15:00,091 source=engine:celery worker=ForkPoolWorker-2 task_id=51e4075b-8b9d-4903-87dd-ba8fcb6cbd1e task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Start sync_users_with_cloud
2022-12-26 12:15:00,101 source=engine:celery worker=ForkPoolWorker-2 task_id=51e4075b-8b9d-4903-87dd-ba8fcb6cbd1e task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Grafana Cloud is not connected
2022-12-26 12:15:00,101 source=engine:celery worker=ForkPoolWorker-2 task_id=51e4075b-8b9d-4903-87dd-ba8fcb6cbd1e task_name=apps.oss_installation.tasks.sync_users_with_cloud name=celery.app.trace level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[51e4075b-8b9d-4903-87dd-ba8fcb6cbd1e] succeeded in 0.010040069995739032s: None
2022-12-26 12:16:00,048 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task sync_users_with_cloud (apps.oss_installation.tasks.sync_users_with_cloud)
2022-12-26 12:16:00,051 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[167b93de-1382-4ad1-9e1d-82e6316d7aa5] received
2022-12-26 12:16:00,053 source=engine:celery worker=ForkPoolWorker-2 task_id=167b93de-1382-4ad1-9e1d-82e6316d7aa5 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Start sync_users_with_cloud
2022-12-26 12:16:00,062 source=engine:celery worker=ForkPoolWorker-2 task_id=167b93de-1382-4ad1-9e1d-82e6316d7aa5 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Grafana Cloud is not connected
2022-12-26 12:16:00,062 source=engine:celery worker=ForkPoolWorker-2 task_id=167b93de-1382-4ad1-9e1d-82e6316d7aa5 task_name=apps.oss_installation.tasks.sync_users_with_cloud name=celery.app.trace level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[167b93de-1382-4ad1-9e1d-82e6316d7aa5] succeeded in 0.00890169599733781s: None
2022-12-26 12:17:00,048 source=engine:celery worker=Beat task_id=??? task_name=??? name=celery.beat level=INFO Scheduler: Sending due task sync_users_with_cloud (apps.oss_installation.tasks.sync_users_with_cloud)
2022-12-26 12:17:00,051 source=engine:celery worker=MainProcess task_id=??? task_name=??? name=celery.worker.strategy level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[0b297fdc-1790-4ef2-976e-07892100cecb] received
2022-12-26 12:17:00,052 source=engine:celery worker=ForkPoolWorker-2 task_id=0b297fdc-1790-4ef2-976e-07892100cecb task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Start sync_users_with_cloud
2022-12-26 12:17:00,063 source=engine:celery worker=ForkPoolWorker-2 task_id=0b297fdc-1790-4ef2-976e-07892100cecb task_name=apps.oss_installation.tasks.sync_users_with_cloud name=apps.oss_installation.tasks level=INFO Grafana Cloud is not connected
2022-12-26 12:17:00,063 source=engine:celery worker=ForkPoolWorker-2 task_id=0b297fdc-1790-4ef2-976e-07892100cecb task_name=apps.oss_installation.tasks.sync_users_with_cloud name=celery.app.trace level=INFO Task apps.oss_installation.tasks.sync_users_with_cloud[0b297fdc-1790-4ef2-976e-07892100cecb] succeeded in 0.0111293520021718s: None
stvnwrgs commented 1 year ago

I have the same error with production EKS setup. After installing and configuring the addon I get the following error message in Grafana after the Sync:

image

The only log message that gets printed out is the following: 

oncall-celery-59ffd8bcd-4nstd 2022-12-28 15:23:12,036 source=engine:celery task_id=acafdef3-a1db-445c-9d79-9ceb5b5ba82c task_name=apps.grafana_plugin.tasks.sync.plugin_sync_organization_async name=apps.grafana_plugin.helpers.client level=WARNING Error connecting to api instance Expecting value: line 2 column 1 (char 1)
mark260486 commented 1 year ago

Same error here!

2023-07-27 10:02:49,045 source=engine:celery worker=ForkPoolWorker-2 task_id=46e819f6-0be7-458d-9d3a-50fdb6ff34ae task_name=apps.grafana_plugin.tasks.sync.plugin_sync_organization_async name=apps.grafana_plugin.helpers.client level=WARNING Error connecting to api instance HTTPConnectionPool(host='localhost', port=3000): Max retries exceeded with url: /api/org (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f07821ffc10>: Failed to establish a new connection: [Errno 111] Connection refused'))

In docker-compose i've tried a different settings: local IP, "host.docker.internal", localhost. Error still the same. изображение

mark260486 commented 1 year ago

I was able to fix this with GRAFANA_API_URL: http://host.docker.internal:3000 in docker-compose.yml and adding 

    extra_hosts:
      - "host.docker.internal:host-gateway"

to Celery container.

aswadsun commented 1 year ago

Got the same error while connecting external grafana to oncall-engine which is installed using helm chart. However, if using provided grafana in the same helm chart, it works.

ESNewmanium commented 1 year ago

Note that you can also get this if you input the OnCall API URL with a trailing slash. Hope this helps someone!

When configured with "http://oncall.dev.example.com/":

2023-09-28 20:24:18 source=engine:app google_trace_id=none logger=root inbound latency=0.021687 status=404 method=POST path=//api/internal/v1/plugin/self-hosted/install user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 content-length=0 slow=0
2023-09-28 20:24:18 source=engine:app google_trace_id=none logger=django.request Not Found: //api/internal/v1/plugin/self-hosted/install

Successful when configured with "http://oncall.dev.example.com":

2023-09-28 20:24:28 source=engine:uwsgi status=201 method=POST path=/api/internal/v1/plugin/self-hosted/install latency=0.320009 google_trace_id=- protocol=HTTP/1.1 resp_size=405 req_body_size=0
2023-09-28 20:24:29 source=engine:app google_trace_id=none logger=root inbound latency=0.330612 status=200 method=POST path=/api/internal/v1/plugin/status user_agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 content-length=0 slow=0 user_id=1 org_id=1 org_slug=self_hosted_org
2023-09-28 20:24:29 source=engine:uwsgi status=200 method=POST path=/api/internal/v1/plugin/status latency=0.331866 google_trace_id=- protocol=HTTP/1.1 resp_size=482 req_body_size=0