search

algolia / hn-search

Hacker News Search
http://hn.algolia.com
Other
546 stars 73 forks source link

hn-search is blocked by Privacy Badger #89

Closed itamarst closed 7 years ago

itamarst commented 7 years ago

Hi,

I use hn-search API to ensure that readers of my blog get redirected to the discussion on Hacker News. Unfortunately, Privacy Badger (https://www.eff.org/privacybadger) decides that this is a privacy violation and blocks the domain. Privacy Badger uses a heuristic to figure out which sites to block.

The solution is either:

  1. Don't have cookies (at least for the API, human-facing pages are fine). Or,
  2. Respect do-no-track, and then tost a specific document in a specific place that is checked by Privacy Badger (https://www.eff.org/privacybadger#faq--I-am-an-online-advertising-/-tracking-company.--How-do-I-stop-Privacy-Badger-from-blocking-me?)

It would be useful to me, and likely others, if HN search API did one or the other option, and it would also increase user privacy.

Haroenv commented 7 years ago

note that after it's fixed, a dnt policy file should be added

If copies of Privacy Badger have already blocked your domain, you can unblock yourself by promising to respect the Do Not Track header in a way that conforms with the user's privacy policy. You can do that by posting a specific compliant DNT policy to the URL https://example.com/.well-known/dnt-policy.txt, where "example.com" is all of your DNT-compliant domains. Note that the domain must support HTTPS, to protect against tampering by network attackers. The path contains ".well-known" per RFC 5785.

redox commented 7 years ago

Don't have cookies (at least for the API, human-facing pages are fine).

Hmmm, but we don't set cookies from what I'm concerned. We do have a Rails proxy on top of our REST API to handle HN-specific things, but this should not add cookies. Does it?

itamarst commented 7 years ago

Might be CloudFlare setting cookies? I'm seeing __cfduid:"dea15aa7138f31c2ba6d447f4bee77d201467572979" in Firefox inspector.

asurak commented 7 years ago

Which version of the Privacy Badger are you using @itamarst ? The old versions were incredibly aggressive but since then it dramatically improved. The latest version is 2017.1.26

itamarst commented 7 years ago

I believe it's the latest one. However, I was using it previously as well, so old version may have blacklisted the site incorrectly. I wiped the settings and will start from scratch and see if it happens with newest version.

However, even if that solves it there is the issue of existing users of Privacy Badger who blacklisted it in the past...

asurak commented 7 years ago

Yes, we've been discussing it directly with EFF because we got marked servers without any cookies. The new versions behave well.

Adding of the dnt-policy.txt is in progress. Once it is added, it can take up to 7 days for the Privacy Badger to unblock the site.

itamarst commented 7 years ago

Thank you!

asurak commented 7 years ago

The DNT policy is now deployed on https://algolia.com/.well-known/dnt-policy.txt

redox commented 7 years ago

Thanks @asurak 👍