Supporting HTTPS when running locally

abidlabs commented 2 years ago

Hi Adi, Thanks for the quick response. So explain in detail, I have made a gradio app and it by default ran on a http connection when I provided my vm ip address in the server_name parameter for the launch function.http://12.13.65.70:3031 (my VM ip address).

But due to company requirements, I have to run this on https. Like https://12.13.65.70:3031. We have a cert and key available but I am not sure how can I pass them to launch function.

I tried wrapping the gradio interface in a flask app and tried to run it by additionally providing ssl_context. But unfortunately I did not work. Can you suggest any idea here?

Originally posted by @vijay2405 in https://github.com/gradio-app/gradio/discussions/370#discussioncomment-2135364

abidlabs commented 2 years ago

Hi @vijay2405, this is now supported in the latest version of gradio on master which takes two arguments directly: ssl_keyfile and ssl_certfile in the launch() method. You can test this by generating a certificate for localhost (e.g. following this guide: https://dev.to/rajshirolkar/fastapi-over-https-for-development-on-windows-2p7d) and confirm this works for you.

Here is some example code:

import gradio as gr

a = gr.Interface(lambda x:x, "image", "image", examples=["lion.jpg"]).launch(
    share=False, ssl_keyfile="key.pem", ssl_certfile="cert.pem")

Which will produce an interface running on: https://127.0.0.1:7860/ for me

We will release a new version to PyPi by the end of this week . If you need this fix before then, please take a look at the contributing guidelines about how to install from the master branch.

Escovilla commented 1 year ago

I encountered an issue with using a self-signed certificate and encountered an SSL error with the message certificate verify failed: unable to get local issuer certificate . However, after seeing https://stefvanlooveren.me/errors/how-fix-sslerrore-requestrequest-requestsexceptionssslerror-httpsconnectionpool. I made a change in the blocks.py file of Gradio by setting verify=False in the requests.get function requests.get(f"{self.local_url}startup-events", verify=False) , and that solved the issue.

abidlabs commented 1 year ago

Thanks @Escovilla we'll make this fix in the library itself

abidlabs commented 1 year ago

Or actually @Escovilla would you like to open a PR with the fix? Would be much appreciated!

abidlabs commented 1 year ago

@Escovilla I got a notification (although I can't see the comment now) with a message from you saying that verify=False is not advisable. Would you like to reshare your solution and then I think we can close this issue?

Escovilla commented 1 year ago

@abidlabs I initially commented that

removing the verify=false in requests is not advisable for production environments, and although I'm currently utilizing it for my local development, it poses a significant risk for live systems if implemented. Regardless I still used verify=false for my local server for local computers to access (web application using a camera)

We could consider adding an option for both self-signed and trusted certificates that would toggle verify=true for trusted certificates and verify=false for self-signed certificates. To avoid compromising live production while having an optionality for developers to use self signed certificates ( to whatever reason ) use https

abidlabs commented 1 year ago

Thanks for clarifying @Escovilla. Perhaps a flag called ssl_verify in the launch() method could control this. I'll look into this, thank you!

Escovilla commented 1 year ago

Right @abidlabs should I PR this implementation ?

abidlabs commented 1 year ago

That would be much appreciated @Escovilla, I'm happy to review

abidlabs commented 1 year ago

Hi @Escovilla did you get a chance to look into this?

Escovilla commented 1 year ago

Hi @abidlabs I apologize for being away. I will review the matter promptly

gitgithan commented 1 year ago

Is there any guide to setting up https for cloud deployment? (not localhost) It seems setting ssl_keyfile="key.pem", ssl_certfile="cert.pem" in gradio is only one part of the puzzle and there are infrastructure level setups required too?

Why is editting blocks.py of gradio required? This issue and https://github.com/gradio-app/gradio/issues/2551 both editted blocks.py but in different ways.

I tried aws ACM + ELB (couldnt point Domain registrar ionos to ELB which required ip address but ELB DNS name was readable words), certbot (couldn't setup apache properly following https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-ubuntu-20-04 maybe due to lack of ufw knowledge), free wildcard SSL that comes with IONOS domain (not sure if should use SSL Certificate, Intermediate Certificate, or PFX file in gradio launch parameter, tried all all failed).

I'm frequently getting errors from ssl.SSLContext saying can't find cert and key pem (looks like must be absolute path? only solves can't find error but other errors follow), or permission denied (when using certbot generated cert+key) or verify failed: unable to get local issuer certificate . for a few cases i forgot.

VikParuchuri commented 1 year ago

@gitgithan The easiest way to do this is:

Setup a Cloudflare account with HTTPS for your domain
Select "flexible" as your cloudflare SSL mode, which means that your server will run in HTTP, but your client can connect in HTTPS (Cloudflare will handle the certificate)
Run gradio normally with port 80, and no SSL certificate, etc.
Add a cloudflare a record dns entry pointing to the IP of your cloud instance

VikParuchuri commented 1 year ago

@Escovilla @abidlabs Just setting verify=False for requests.get isn't enough to run with a self-signed certificate with the queue enabled and iterative output, since websockets do their own SSL check. I'm not sure how to disable that one (and you probably wouldn't want to!).

gitgithan commented 1 year ago

@VikParuchuri Nice to see you here. Run gradio normally with port 80, and no SSL certificate Whats the importance of starting with server_port=80? Is it to save the effort setting up reverse proxy (which helps user avoid appending :7860 to url)? I'm getting OSError: Port 80 is in use. If a gradio.Blocks is running on the port, you can close() it or gradio.close_all().

When i try to enter https://...:7860 in url i get WARNING: Invalid HTTP request received. . I changed nameservers to the two from cloudflare, set flexible SSL, turned on universal SSL of cloudflare, is there any missing step? I see alot of settings under SSL>Edge certificates (Always Use HTTPS,Opportunistic Encryption,Automatic HTTPS Rewrites), the last of which links to configuration rules, I tried adding a rule (not sure why this page shows almost same settings as Edge certificates page, does this mean there are 2 levels of enabling required for features?) and still both http and https don't work when im running gradio on default 7860 without specifying server_port=80 or the 2 ssl parameters. Top of edge certs page shows universal ssl is active on *.cloudarchitect.online, cloudarchitect.online.

What's the relationship between flexible mode (https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/flexible/) and the settings on edge certificates? Is it one automatically sets the other, or i have to manually get the right toggle combination on edge certificates for flexible mode to work?

I'm deploying on EC2, are there cloudflare ip addresses that must be added to security groups? I added all these https://www.cloudflare.com/en-gb/ips/ still don't work search.cloudarchitect.online (both http https) shows website down when gradio is running on default 7860 and doesn't respond (blank screen) when append :7860

Would you say it's a bad idea to start with cloudflare if i don't have strong undestanding yet of SSL or general networking knowledge? The classic it works i don't know why it doesn't work i don't know why. New problem comes, start from 0 again. Now that i've swapped nameservers to cloudflare, looks like http://search.cloudarchitect.online:7860 thats previously working on ionos nameserver is now unavailable so I can't test whether changes i make in app or any of the supporting infrastructure works. Any way to turn off cloudflare effects while still using cloudflare nameserver? Goal is to recover http://search.cloudarchitect.online:7860 and be able to test other parts

VikParuchuri commented 1 year ago

@gitgithan Port 80 is the typical HTTP port. Yes, if you're running a reverse proxy like nginx, you can specify another port, as long as nginx is configured to map port 80 to that port. I mentioned port 80 because you want to run gradio in http mode, without any ssl cert, versus https, which would be port 443.

I think you have a few separate issues. The reason cloudflare isn't working is because your web server isn't running properly. I would resolve each one separately, instead of together.

First, run your gradio server on port 80 (or whatever port your reverse proxy is configured to forward to). Set share=True in launch, and make sure you can access the server from the huggingface url that is generated.
Then, make sure your EC2 firewall settings, etc, enable you to access gradio on port 80. Try to access gradio using the instance ip.
Then, setup cloudflare ssl. You just need to add a DNS entry to your EC2 instance up, as well as the SSL stuff.

gitgithan commented 1 year ago

Answering my previous question, none of Edge certificates (Always Use HTTPS,Opportunistic Encryption,Automatic HTTPS Rewrites) need to be toggled on.

@VikParuchuri

you want to run gradio in http mode

Is that done by demo.launch(server_port=80)? I can't launch it on 80, getting OSError: Port 80 is in use. If a gradio.Blocks is running on the port, you can close() it or gradio.close_all(). lsof don't show anything on 80 too.

So I stopped experimenting on 80 and used default 7860 (by not specifying server_port). Later i found this page (https://developers.cloudflare.com/fundamentals/get-started/reference/network-ports/) that prompted me to choose one of the https ports (8443) to experiment after 7860 got nowhere.

I don't understand how the ports on this page relate to the 4 Cloudflare SSL/TLS options, like what is cloudflare is actually doing or can't do (and why) when I use a port on/not on that page. I always thought port number had nothing to do with http or https but it seems cloudflare makes an association there that makes or breaks things.

Results

Proxy On/Off (gray/orange cloud)	SSL Mode	Gradio Port	http/https with domain name requested from chrome	Results
gray	Off	7860	http	Not secure Lock Icon but can use app
gray	Off	7860	https	This site can't provide a secure connection. (Black screen Nothing shown)
gray	Off	8443	http	Not secure Lock Icon but can use app
gray	Off	8443	https	This site can't provide a secure connection. (Black screen Nothing shown)
orange	Off	7860	http	Stuck at Pending
orange	Off	7860	https	Stuck at Pending
orange	Off	8443	http	400 Bad request the plain http request was sent to a https endpoint (cloudflare message)
orange	Off	8443	https	Secured Lock, all works with webcam
orange	Flexible	7860	http	Stuck at Pending
orange	Flexible	7860	https	Stuck at Pending
orange	Flexible	8443	http	400 Bad request the plain http request was sent to a https endpoint (cloudflare message)
orange	Flexible	8443	https	526 Invalid SSL Certificate

To prevent false negative, server was checked to be working on both 7860 and 8443 using ip instead of domain name so not depending on cloudflare dns

Surprises

The only row that works with secured lock uses SSL Mode off (i forgot to set to flexible then). How did it even get the secured lock? I would think SSL Mode Off means https is disallowed/disabled, even if it is allowed/using SSL, mode off implies no certs are used so where did the secure lock come from. It's as if SSL Mode Off was Flexible. (Maybe it takes more than half day for any change in SSL Mode to take effect, not knowing when this happens or what mode im currently on makes observations hard to interpret)
526 in last row. The cert is the universal cert from cloudflare itself so least expect cloudflare to be saying its own cert is invalid. It's as if Flexible is Full (Strict) since https://support.cloudflare.com/hc/en-us/articles/115003011431#526error says in 2nd condition:
```
Error 526 occurs when these two conditions are true:
```

Cloudflare cannot validate the SSL certificate at your origin web server, and Full SSL (Strict) SSL is set in the Overview tab of your Cloudflare SSL/TLS app.



I also tried nginx on both SSL modes (off and flexible) `listen 80`, `proxy_pass http://localhost:8443;` which worked on both http and https (so 4 permutations) without me typing :8443 (http no webcam https webcam ok )
I was swapping SSL Mode between off and flexible quickly and the changes were instantaneous. Off makes https give not secure icon and flexible gives https the secure icon. http works on both modes with no secure icon. 

Also i had to access https://search.cloudarchitect.online in chrome incognito to get secure icon. Not incognito https did work once during my SSL mode quick swapping experiments but now keeps redirecting me to insecure http version somehow even after i clear all cookies. Had to use a new chrome profile to get secure icon if not incognito. In old profile, it  also worked after clearing the default checked items (browsing history, download history, cookies and other site data, cached images and files). Not sure what kind of chrome caching causes a https to redirect to http

**Notes**
1. https://community.cloudflare.com/t/why-flexible-ssl-mode-is-not-the-best-choice/63531

abidlabs commented 1 year ago

Just circling back, it sounds like a setting to disable ssl verify would help with the local deployment. Is that something you are still interested in implementing @Escovilla?

Taited commented 1 year ago

https://dev.to/rajshirolkar/fastapi-over-https-for-development-on-windows-2p7d

This is also what I am interested in.

papuSpartan commented 1 year ago

@abidlabs Would it not be possible to pass an ssl context to the websocket creation at the /queue/join route in routes.py? This may be doable using the tls arguments already implemented for the uvicorn setup. Passing self-signed certificates using REQUESTS_CA_BUNDLE was enough before for getting https up and running locally. Now, the wss connection failing halts the entire flow of the gradio runtime. Probably just need to set verify=False in https://github.com/gradio-app/gradio/pull/3873

bharadwaj509 commented 1 year ago

So the https issue is resolved? Could some one let me know?

garrettsutula commented 1 year ago

Thanks for clarifying @Escovilla. Perhaps a flag called ssl_verify in the launch() method could control this. I'll look into this, thank you!

I added what I think needs to be done here in PR 3873

@papuSpartan Not sure it's 100% related - I think all that really needs to be done w/ that is to trust the self-signed cert in the host OS of the client device.

garrettsutula commented 1 year ago

@garrettsutula It would be, intuitively, except that the created wss websocket does not respect system-level trust store certificates. This is also why I had to use REQUESTS_CA_BUNDLE to get HTTPS connections to form properly when passing a certificate with the tls command line args (requests doesn't respect them either). It will pass your cert but doesn't do anything extra in the case that it is a self-signed certificate. Exposing an option to disable verification with ssl_verify is helpful but it doesn't have the same level of confidentiality as providing the extra context of that one cert to the wss websocket.

Looks like this is a workaround: https://github.com/gradio-app/gradio/commit/417004f9465aa5182b74093d0f245b4f76159864

The problem is that the http client used in queueing.py isn't being initialized with any ssl context.

I am not sure how best to pass down from blocks.py - I could kludge it and add the ssl params to the Queue constructor but there's probably something different/better that could be done instead.

ucas010 commented 1 year ago

have the same bug and problem

qiyuangong commented 1 year ago

@garrettsutula It would be, intuitively, except that the created wss websocket does not respect system-level trust store certificates. This is also why I had to use REQUESTS_CA_BUNDLE to get HTTPS connections to form properly when passing a certificate with the tls command line args (requests doesn't respect them either). It will pass your cert but doesn't do anything extra in the case that it is a self-signed certificate. Exposing an option to disable verification with ssl_verify is helpful but it doesn't have the same level of confidentiality as providing the extra context of that one cert to the wss websocket.

Looks like this is a workaround: 417004f

The problem is that the http client used in queueing.py isn't being initialized with any ssl context.

I am not sure how best to pass down from blocks.py - I could kludge it and add the ssl params to the Queue constructor but there's probably something different/better that could be done instead.

https://github.com/gradio-app/gradio/pull/3873 save one day for me. :)

Unfortunately, we still need to set blocks.ssl_verify = False to avoid blocks raise [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129) Error.

Instead of changing any code in Gradio, you can set blocks.ssl_verify = False before creating app.

# This line turn off ssl_verify for blocks
blocks.ssl_verify = False

gradio_app = App.create_app(blocks)

miguelwon commented 3 months ago

I'm having a related issue. I also need https, and I'm able to run my app with:

HOST = "0.0.0.0"
PORT = 8010
demo.queue()
demo.launch(
    favicon_path=favicon_path, 
    server_name=HOST, 
    server_port=PORT,
    share=False,
    ssl_keyfile="key.pem", ssl_certfile="cert.pem",
    ssl_verify=False)

But although the app is running in a remote VM and I can't reach it. I'm getting no response. But the moment I remove ssl:

HOST = "0.0.0.0"
PORT = 8010
demo.queue()
demo.launch(
    favicon_path=favicon_path, 
    server_name=HOST, 
    server_port=PORT,
    share=False,

I get a response and it works nicely. Appreciate any help.

gradio-app / gradio

Supporting HTTPS when running locally #563