search

privacytests / privacytests.org

Source code for privacytests.org. Includes browser testing code and site rendering.
https://privacytests.org
MIT License
846 stars 25 forks source link

test arkenfox.js #80

Open arthuredelstein opened 2 years ago

Thorin-Oakenpants commented 2 years ago

FYI - https://news.ycombinator.com/item?id=29850780

Interesting that you would want to test AF. I already know what it would be. If you want to do it correctly ... it's arkenfox with uBO + two filters added :) .. and I only just switched if from FPI to dFPI (strict) when v96 is final

It should pass (here's my prediction)

It's slightly hardened, and not everyone can live with it like that, mostly referrers

We gonna need a bigger set of tests :) What do you suggest ?

edit: updated filters link

Thorin-Oakenpants commented 2 years ago

just to be clear, I think adding AF is a waste of time, but feel free to do a one off test for me :)

We gonna need a bigger set of tests :) What do you suggest ?

I have some ideas

arthuredelstein commented 2 years ago

just to be clear, I think adding AF is a waste of time, but feel free to do a one off test for me :)

Why do you think it is a waste of time? I was thinking it would be an interesting thing to show alongside various extensions.

* under navigation test? beacon, link-pre-fetching, predictor

* sanitizing: e.g. cache/HSTS etc, cookies + site data, etc

* a "security" section: e.g. the five safebrowsing types

These are good ideas. Thanks!

Thorin-Oakenpants commented 2 years ago

These are good ideas. Thanks!

I just cherry picked a few things from AF's prefs that I know it will pass that many other forks/etc won't 😀

Why do you think it is a waste of time?

Because we basically already know the answers - see my prediction. In the current tests, AF96 is almost (95%) the same as LW (I made LW have an intervention for their own health and then spent the last 8 months helping one of their devs with info and suggestions). About the only diff is referrers in the current tests.

If you add the things that AF will pass, it would make me look like a genius - and this includes more diffs from LW (but probably not for long if they would only listen to me on the last few items). So eventually, you're just going to end up with LW and an extra tick or two

Sure it's interesting. IDK how you want to frame it. A new set of "custom" browser results makes sense i.e not a fork and not default. IMO, it should be arkenfox with uBO + 2 filterlists (but this will become immaterial)

and arkenfox on it's own is not an "extension" or a fork. IDK how/where you would fit it in, plus we already know the results

Up to you :) I'm more concerned with how others read the results - because without uBO + extras, it's not what I suggest. On the other hand the AF user.js is a "template" that flips 150 prefs (and essentially warns against the rest), of which about 10 prefs (highlighted and suggested to change if needed) cover 99% of the breakage, and users are going to change some of those. So it's not real world: there is no default

d4rklynk commented 2 years ago

Up to you :) I'm more concerned with how others read the results - because without uBO + extras, it's not what I suggest. On the other hand the AF user.js is a "template" that flips 150 prefs (and essentially warns against the rest), of which about 10 prefs (highlighted and suggested to change if needed) cover 99% of the breakage, and users are going to change some of those. So it's not real world: there is no default

But in LW there are some settings you can flip too. I think adding AF is a really good idea. You remove the top 1 issue of AF (I think it's 1601) and you good to go. I don't think you should test any browser with any add-on though. Just test AF with the 1601 fix and that's all

xe-3 commented 1 year ago

I definitely think there would be value in testing and listing Arkenfox.

There are a lot of new, uninformed, or non-techie users that get directed to the privacytests website (mostly directed there by people promoting Brave or occasionally Librewolf). These people often come away with the impression that Firefox is not very private which couldn't be further from the truth, a well configured Firefox (such as Firefox + Arkenfox + uBO) can be very privacy preserving.

I disagree with the person above me who thinks extensions shouldn't be included. In general I think they are correct, but in the case of Arkenfox nobody using Arkenfox isn't also using uBO as recommended.

I hope that AF gets added

on a separate note, is it possible for us to test our own browsers?

Thorin-Oakenpants commented 1 year ago

from https://github.com/privacytests/privacytests.org/issues/80#issuecomment-1008282746

I have some ideas

  • sanitizing: e.g. cache/HSTS etc, cookies + site data, etc
    • will look impressive for AF/LW and of course TB (being in PB Mode)

I can't believe it's only been 18 months, time flies ... how did my prediction stack up? :) thanks Arthur for the tests

woo

arthuredelstein commented 1 year ago

from #80 (comment)

I have some ideas

  • sanitizing: e.g. cache/HSTS etc, cookies + site data, etc

    • will look impressive for AF/LW and of course TB (being in PB Mode)

I can't believe it's only been 18 months, time flies ... how did my prediction stack up? :) thanks Arthur for the tests

woo

from #80 (comment)

I have some ideas

  • sanitizing: e.g. cache/HSTS etc, cookies + site data, etc

    • will look impressive for AF/LW and of course TB (being in PB Mode)

I can't believe it's only been 18 months, time flies ... how did my prediction stack up? :)

Perfect! :) And, yeah, I can't believe it's been 18 months either...

Thorin-Oakenpants commented 1 year ago

OT: hmm, I noticed FF PB mode failed cross-session 3p prefetch cache - did you want to alert someone at Mozilla (or do you want me to try to do it), or is it a bug

update: fyi it is being handled upstream

opusforlife2 commented 3 months ago

I think it's still worth it to have Arkenfox'ed Firefox as a separate browser there.

From https://librewolf.net/license-disclaimers/ : "This also isn't Arkenfox... We rely heavily on Arkenfox’s expertise, research, and knowledge, but we choose our own default preferences configuration."

So LW works as a very rough substitute for getting an idea about AF, sure, but it would be so much better to have it there directly.

After all, Mull is present in the Android section, and it uses Arkenfox directly.

Thorin-Oakenpants commented 3 months ago

no thanks, please don't highlight/promote arkenfox via privacytests - I'm trying to kill it

opusforlife2 commented 3 months ago

I'm trying to kill it

Wot

Thorin-Oakenpants commented 3 months ago

https://github.com/mullvad/mullvad-browser/issues/1#issuecomment-1495998714

https://github.com/arkenfox/user.js/issues/1842#issuecomment-2153421218

basically this comes down to getting TB and MB to relax disk usage when the user opts into it - basically I think both should start in persistent mode but sanitize, but there's a lot of work to do before we get to that

opusforlife2 commented 3 months ago

Alright, got it. This issue should probably be closed then.