search

andybalholm / redwood

Web content filter that runs as an HTTP proxy
BSD 2-Clause "Simplified" License
225 stars 37 forks source link

Help allowing *some* youttube videos only #61

Closed HiranChaudhuri closed 3 years ago

HiranChaudhuri commented 3 years ago

So having fixed the TLS certificate I can now see redwood in action. Having started off with the example configuration from https://github.com/andybalholm/redwood-config I can see that all of youtube is getting blocked. This is what I intend in general anyway. However teachers tend to upload their video content to youtube, and then the kids shall be able to access those videos.

So what I am trying to achieve is

Here is how I created that new category and also added the referrer acl:

$ cat categories/school/category.conf 
description: Arbeitsmaterial für die Schule
action: allow
$ cat categories/school/sites.list 
padlet.com/la_jung/Bookmarks 500
padlet.com 300
www.youtube.com/watch.v=QXS4exogzb0 300
$ cat acls.conf 
acl lan-ip user-ip 192.168.0.0/16 10.0.0.0/8 127.0.0.0/8 172.16.0.0/12
require-auth !lan-ip

acl connect method CONNECT
allow connect updates
ssl-bump connect

acl css content-type text/css
allow css

acl school referer padlet.com

acl text content-type text/*
acl text content-type application/javascript application/x-javascript application/json
acl text content-type application/xhtml+xml application/xml application/rdf+xml application/rss+xml application/atom+xml application/emma+xml
phrase-scan text
$

What am I missing?

HiranChaudhuri commented 3 years ago

I think I was able to resolve the whitelist part. While youtube is still blocked as part of video-sharing, I renamed my new category 'school' to '01_school' so it appears earlier in redwood's scan. Now it seems to play a role and the category and it's score is evaluated.

So it seems I have three out of the four items I listed above running. Still missing is how I can use the referer to automatically make all teacher's choices accessible.

HiranChaudhuri commented 3 years ago

Update: Now that the category is renamed, also the other bookmarks seem to magically work. So it seems the rule I had added before 'acl school referer padlet.com' was correct and now it is active.

Nothing else remaining, so we can close this issue.

Redwood is a really nice project. Thank you for sharing.. :-)

elico commented 3 years ago

@HiranChaudhuri Just pay attention that redwood url rules are not case sensitive. The next url: www.youtube.com/watch.v=QXS4exogzb0 300 is the exact as: www.youtube.com/watch.v=qxs4exogzb0 300

but you might be able to match a url with a regex. I know you can do that for the host.

hiranchaudhuri1 commented 3 years ago

Good point, @elico. This is mentioned somewhere in the documentation but actually resembles a huge pitfall.

elico commented 3 years ago

So having fixed the TLS certificate I can now see redwood in action. Having started off with the example configuration from https://github.com/andybalholm/redwood-config I can see that all of youtube is getting blocked. This is what I intend in general anyway. However teachers tend to upload their video content to youtube, and then the kids shall be able to access those videos.

So what I am trying to achieve is

* block all youtube per default

* allow access to the teacher's bookmarks page
  While creating an additional category for 'school' I have success as the bookmarks page is accessible now and is categorized as 'school' (and as I expected).

* allow videos that I add to a whitelist
  When I add youtube videos they still are blocked afterwards - the school category seems to not matter at all.

* allow videos that are accessed through the teacher's bookmarks page on a specific URL
  Here I was hoping to allow all requests that come with a referer containing the teacher's bookmarks URL, but that seems to not work either for me

Here is how I created that new category and also added the referrer acl:

$ cat categories/school/category.conf 
description: Arbeitsmaterial für die Schule
action: allow
$ cat categories/school/sites.list 
padlet.com/la_jung/Bookmarks 500
padlet.com 300
www.youtube.com/watch.v=QXS4exogzb0 300
$ cat acls.conf 
acl lan-ip user-ip 192.168.0.0/16 10.0.0.0/8 127.0.0.0/8 172.16.0.0/12
require-auth !lan-ip

acl connect method CONNECT
allow connect updates
ssl-bump connect

acl css content-type text/css
allow css

acl school referer padlet.com

acl text content-type text/*
acl text content-type application/javascript application/x-javascript application/json
acl text content-type application/xhtml+xml application/xml application/rdf+xml application/rss+xml application/atom+xml application/emma+xml
phrase-scan text
$

What am I missing?

I will to make your life a bit easier. To block a specific video you need to block 3 or more youtube patterns with the ID/ The youtube image, the video main link, the embedded version and maybe couple others. To write a good acl for your use case you will need to write two different acl sets. The first is the rules to allow and then the rules to block/deny. You will need to first validate the allowed list since the blocked would be "all". With that pay attention that you must use some level of either coding or regex to do that. I have not tried it but you will need to allow: www.youtube.com/watch?v=qxs4exogzb0 and block www.youtube.com/watch?v=*

In my test results I found it very easy to write an external classifier even in PHP compared to writing the acl rules. I am using mysql at the moment in a key value similar format which works pretty well.

I have also seen you have tried my yt-classficiation example which is great as an example. Don't use referrer.... You better give the teacher access to add videos using some javascript bookmarklet or form.