search

serverless-dns / blocklists

An opinionated collection of blocklists for RethinkDNS.
https://rethinkdns.com/configure
Mozilla Public License 2.0
85 stars 24 forks source link

OISD NSFW not working #164

Closed anxiouslelimo closed 2 weeks ago

anxiouslelimo commented 1 year ago

Even after enabling oisd nsfw blocklist adult sites are loading. Verified that the sites are included in oisd list.

anxiouslelimo commented 1 year ago

In addition enabling oisd nsfw blocking legitimate sites like whatculture.com

anxiouslelimo commented 1 year ago

Issue not present anymore. Closing

ignoramous commented 1 year ago

Thanks for the confirmation! (:

I remember looking at this a week ago and wondering where the issue really was. Looks like it isn't present anymore. One of the recent changes must have fixed this.

anxiouslelimo commented 12 months ago

@ignoramous how often is the blocklist updated? I reported a nsfw site and it was added to oisd nsfw list today, but the site is still accessible via rethinkdns.

ignoramous commented 12 months ago

Every 8th day of the month (automated):

anxiouslelimo commented 12 months ago

Thanks @ignoramous.

anxiouslelimo commented 10 months ago

@ignoramous Site blocked in sky is not blocked in max. eg: aagmaal.lat. Also a site I reported and blocked in oisd nsfw 10 days ago i.e; before the last blocklist update is still accessible through both the resolver. eg: sxyprn.unblockit.foo. I will keep a watch on sky as it will be updated today.

ignoramous commented 10 months ago

sky is subject to heavy DoS attacks, and it may behave differently. We are working on a new architecture to bring down the costs for sky.

As for max, it continues to function but our infrastructure provider has trouble keeping up the amount of requests we are sending their way. Of late, automated blocklist updates have been failing (again), too. I'll look into them today and get back to you.

anxiouslelimo commented 10 months ago

@ignoramous Site blocked in sky is not blocked in max. eg: aagmaal.lat. Also a site I reported and blocked in oisd nsfw 10 days ago i.e; before the last blocklist update is still accessible through both the resolver. eg: sxyprn.unblockit.foo. I will keep a watch on sky as it will be updated today.

sky and max same result

anxiouslelimo commented 10 months ago

Of late, automated blocklist updates have been failing (again), too. I'll look into them today and get back to you.

Hi @ignoramous do we have an update on this?

ignoramous commented 10 months ago

Since Nov 18, We're locked out of some of the Cloudflare services due to 'trust and safety' issue for which we have received no answer, as of today. Blocklist builds are working, but the uploads and downloads to Cloudflare R2 (blob store) is not. The alternative is to use S3, but we moved to R2 because hosting on S3 costs us $500/mo compared to $5/mo on R2.

ignoramous commented 10 months ago

The blocklist updates are up and working (for now); the latest blocklist version is 1701089546918.

anxiouslelimo commented 10 months ago

@ignoramous Site blocked in sky is not blocked in max. eg: aagmaal.lat. Also a site I reported and blocked in oisd nsfw 10 days ago i.e; before the last blocklist update is still accessible through both the resolver. eg: sxyprn.unblockit.foo. I will keep a watch on sky as it will be updated today.

@ignoramous Still sites unblocked in max

ignoramous commented 10 months ago

@ignoramous Still sites unblocked in max

Thanks.

Our weekly automated deploys (which also updates blocklists) to our resolvers on Fly.io (max) failed (ref). I've re-ran it so it'd pickup the latest blocklists, just now.

anxiouslelimo commented 10 months ago

Sites blocked in max. Added new sites to OISD NSFW yesterday. Will check and revert after the next scheduled automatic update.

ignoramous commented 10 months ago

In case you didn't know, anyone can check the latest blocklist update time and search inside blocklists for domains on our website; ex: https://rethinkdns.com/search?q=segment.io

The next auto-update is scheduled for 1 Dec at ~22:00 (UTC) (and max should pick it up 24h later once automated tests are complete).

btw, we are working on daily blocklist updates. Hopefully, it is up by the end of this year.

anxiouslelimo commented 10 months ago

@ignoramous https://github.com/serverless-dns/serverless-dns/actions/runs/7068940824 failed

ignoramous commented 10 months ago

Your account has overdue invoices.

Thanks. Ah well... will re-run once that's sorted (:

ignoramous commented 2 weeks ago

OISD (Adult) is loading just fine.

Sky is facing huge amounts of traffic ($8000+ per month) and so it is run in "restricted" mode to keep the costs in control, as a result of which not ALL blocklists are applied (only a few good trackerware/spyware/family/malware ones are). Max however continues to run at full-capability.

We'll probably get Sky sorted sometime later this year (but that's a separate issue unrelated to this one). Closing... (but feel free to re-open in case the original issue of OISD (Adult) not working is still a problem).