http:// block is limited to snap.berkely.edu

[menasheh]

The (http:// []) block reports an empty string for any site except for snap.berkeley.edu. The browser does note the cross-site requests, though.

[jmoenig]

the HTTP block only works with CORS compliant web services. You can use services like corsproxy.com to scrape other websites. The great thing about the HTTP block is that you can use it for your own web services (the ones where you control the server yourself)

[nathan]

Could we check for a security error and automatically use corsproxy to send non-cookied requests to domains that don't enable CORS?

[brianharvey]

On 5/6/13 12:33 PM, Nathan Dinsmore wrote:

Could we check for a security error and automatically use corsproxy

corsproxy embeds something from jsfiddle.net which flunked Google malware testing:

http://www.google.com/safebrowsing/diagnostic?site=jsfiddle.net

I think we've already had this conversation, but it'd be better if we could do what corsproxy does ourselves rather than rely on a third party.

[nathan]

corsproxy embeds something from jsfiddle.net which flunked Google malware testing

As an example of how to use it, not as part of the service itself. But yeah, you could set up your own pretty easily.

[achalddave]

I'm not sure if this is still a concern, but I thought we didn't want to implement something like CORS proxy because it could be used to circumvent school/workplace restrictions on websites. Is this still a concern? If not, implementing CORS proxy is not too difficult.

[menasheh]

I happen to have a web filter which blocks proxies, so that doesn't really help me. What exactly is the point of CORS anyway?

[mcookAmazon]

Hi,

Not sure how to use the corsproxy feature. Do I need to set up my own instance of SNAP on a web server? Can I somehow try to invoke a web service using the online version of SNAP, hosted at http://snap.berkeley.edu/?

Mike

[joefarebrother]

Would there be a way to implement a cors proxy in your server but have it so it can only be used by snap not to directly bypass school filters?

[xtitter]

We can definitely write a service on snap.berkeley.edu that allows Snap programs to ignore CORS issues when it makes http:// requests. (I'm not sure whether it would solve the dropping-images problems, though).

Effectively, you can do this with corsproxy, but its slow and not something we should trust in general (i.e., it could log all the data that you or your students get through it, etc).

We certainly wouldn't want to build in use of corsproxy into Snap itself; whether we would want to build in use of a snap.berkeley.edu hosted service is another question we could debate. It is nice that snap is a 100% client side solution, making it easy to host elsewhere (github pages, for instance), but this probably isn't important to 99% of users. And, I believe we do have dependencies on snap.berkeley.edu currently (specifically, the cloud storage only works for snap.berkeley.edu hosted programs, I think?).

If someone is willing to write/find a simple script (python 2.4, unfortunately, or php) that we all could agree will work, that would be a great start.

[joefarebrother]

Simple PHP one-liner: echo file_get_contents("http://" . $_GET["url"]);

Then load snap.Berkley.edu/path/to/the/file.PHP?url=some.website.com/page

And schools couldn't use it to bypass filters because the browser would try to load CSS files and SWF files relative to snap.Berkley.edu so they won't load. Of course, you can translate it into whatever language the server is written in.

[jmoenig]

So, are there any plans to implement a corsproxy ourselves? Anyway, I'm closing this issue since the HTTP block is, in fact, not limited to snap.berkeley.edu

[xtitter]

"plans" is an interesting word, lol.

I'm happy to install a php (2.5.1) or python (2.4) based corsproxy in the right place on snap.berkeley.edu (how about in /proxy/cors.php) and seeing that it at least doesn't throw a server error (not trivial on the apps.eecs development environment).

But I'm not confident at making it bug or security-hole free. I'm not sure that it circumvents any school/workplace restrictions on webaccess -- since a student could only work from Snap! and since they could use www.corsproxy.com/ anyway -- but I don't really know.

If we write a local version:

Do we want to restrict it to requests from javascript that came from snap.berkeley.edu? Would that hinder development? (we could add github, for github-pages hosted snaps that can't run the proxy script, and force localhosted snaps to run the script locally).

Do we want to restrict it to requests to certain servers? Probably not. Or, to blacklist certain server targets?

http only, right, not https? But then https://github.com/jublonet/codebird-cors-proxy/ says never use http. Their corsproxy is not a simple one liner, btw, even allowing for all the twitter-specific code.

I don't have a lot of confidence about most of these issues. Can we chat with some security-minded person, Brian?