search

trademapper / trademapper-js

JS libraries for mapping trade using d3.js
Other
26 stars 18 forks source link

corsproxy and https #43

Open foobacca opened 9 years ago

foobacca commented 9 years ago

@willowo tried to share a URL using a CSV she'd put in dropbox. The URL of the CSV looked like https://www.dropbox.com/s/<randomstring>/comptab_2014-11-05%2017-23_comma_separated.csv - the important part being the https at the start of the URL. I found the link above gave me a download page, but if you add ?dl=1 to the end of the URL you get the actual file.

However corsproxy doesn't really do https in a way that works for us. https://www.corsproxy.com/ is available, but the certificate is not valid, so you need to add the certificate to your browser. And then I could get another CSV working over https, but not from dropbox.

We could host our own corsproxy service I guess - there are some open source ones around - but that needs a little more work.

@paulmr - do you have any time to look at this. Maybe there's something I've missed.

paulmr commented 9 years ago

Hi @foobacca , yeah I think your assessment is correct. There is no way (that I can find anyway) to indicate to corsproxy.com that it needs to download the resource via https (as you don't include the protocol when you generate corsproxy.com/.... URL) so I guess it always accesses that resource via plain HTTP. It looks like if you try to access a dropbox link via HTTP you get a redirect response to HTTPS, which is probably ignored by corsproxy:

$ curl -I http://www.dropbox.com/s/irqkrjhqr1v6hsz/cites_unicorn.csv?dl=1
HTTP/1.1 301 Moved Permanently
[...]
Location: https://www.dropbox.com/s/irqkrjhqr1v6hsz/cites_unicorn.csv?dl=1

I kind of feel like this is more of a hosting issue rather than a TradeMapper one, as if we keep TradeMapper as a client app, then any server issues are strictly related to the site that is hosting it. We should probably allow configuring the Cors Proxy URL though, so that if a hosting site does want to create their own proxy they can. I am half tempted to make a generic proxy and stick it on an app engine instance somewhere! But I am not sure if I am going to get the time for that.