grafana / oncall

Developer-friendly incident response with brilliant Slack integration
GNU Affero General Public License v3.0
3.51k stars 291 forks source link

Unable to use a http proxy for oncall backend #1155

Open anilkapu opened 1 year ago

anilkapu commented 1 year ago

I have a need to run the on-call backend in an environment which is behind a corporate firewall which doesnt have connectivity to internet. So for any outbound traffic I need to use a http proxy. When I'm using Outgoing Webhook feature to send alert notification to external app like slack or google chat over the internet I need to use a http proxy. But I see that even after configuring the outgoing webhook and selecting the same in escalation policy, alert notification is not triggered. But in the Alert Timeline I can see the message "outgoing webhook *** triggered by escalation chain". I cannot see any errors in either engine container logs or celery task logs

I see neither UI plugin nor in the on-call backend code there is a provision to specify HTTP PROXY.

It would be really helpful if the application allow to use a HTTP proxy to as this I guess is a very common scenario in secure setup

I guess you can add something like

if self.proxy:
  post_kwargs["proxies"] = {"http": <http url> , "https": <https url>} 

in https://github.com/grafana/oncall/blob/5f69350679d2dc940e10b25865d1194c2d632a88/engine/apps/alerts/models/custom_button.py#L103 this code block

p6297 commented 1 year ago

hey there, I want to work on this issue. can you give me some tips how to solve this issue? i am a newcomer in open source. please educate me a little bit on this issue. thank you

anilkapu commented 1 year ago

hi @p6297 the issue is that, currently there is no option to configure http proxy for the oncall backend. If I deploy the backend in a kubernetes cluster where there is no outbound internet traffic and instead have to use a http proxy to make outbound calls, features like slack notification or google chat notifications through outgoing webhook calls dont work.. Let me know if you need any further info

anilkapu commented 1 year ago

I guess we need a way for the backend to accept HTTP_PROXY environment variable and use it in the code to allow connectivity for outbound traffic

alexintech commented 1 year ago

Hi @anilkapu! You can set proxy configuration in helm values.yml like this:

env:
  http_proxy: "http://<proxy-url>"
  https_proxy: "http://<proxy-url>"
  no_proxy: "127.0.0.1,localhost,.svc,...."

All requests except specified at no_proxy will go through proxy.

kosfango commented 2 weeks ago

Hi @anilkapu! You can set proxy configuration in helm values.yml like this:

env: http_proxy: "http://" https_proxy: "http://" no_proxy: "127.0.0.1,localhost,.svc,...." All requests except specified at no_proxy will go through proxy.

Unfortunately, it doesn't work. Oncall settings show this error diring attempt to configure telegram: (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f0e4d045be0>: Failed to establish a new connection: [Errno 101] Network unreachable'))

Proxy settings present into env.