search

arkenfox / user.js

Firefox privacy, security and anti-tracking: a comprehensive user.js template for configuration and hardening
MIT License
10.03k stars 515 forks source link

recursion testing #1789

Closed Thorin-Oakenpants closed 8 months ago

Thorin-Oakenpants commented 9 months ago

test: https://arkenfox.github.io/TZP/tests/recursion.html

example results: you can ignore iframe and worker

level: 39289
stack: 8448

I'm determining if this (level) is stable enough as a fingerprint, and it seems so (we can round the number into buckets). So the next question becomes what does it return on different hardware. The other factor is enabling/disabling ion/jit etc - which AFAICT requires a restart for it to affect the results (I always close and re-test in a new tab)

So here's some Tor Browser results to give you an idea

recursion level (TB unless stated)

                     worker         doc/iframe
                standard  safer   standard   safer

    win11 32bit   16180    4616     43439    12408 (i686-w64-mingw32 - on 64bit OS)

    win11 64bit   21620    6486     39376    11811 (x86_64-w64-mingw32)
    win11 64bit   21558    6472     39251    11778 (FF123n)
    win10 64bit   21572    6474     39309    11784 (iam-py-test FF121)
 debian12 64bit   21607    6482     39386    11816 (VM on windows)
         debian   21607    6482     39386    11816 (PieroV)
debian bookworm   21551    6465     39358    11807 (2glops FF121)
 fedora39 64bit            6474              11833 (rusty Fedora 39; x86_64; FF121)
 fedora39 64bit   21567    6470     39466    11840 (jayna37 FF121 M2 MacBook Air aarch64)
     arch 64bit   21560    6468     39363    11811 (therealmate FF121)
    LMDE6 64bit   21551    6465     39358    11810 (g-2-s FF121)
    LMDE6 64bit   21607    6482     39386    11816 (g-2-s)

     iMac 64bit   21690    6507     39398    11820 (Apple M1)
    MacBook Pro    6495    6495     11808    11808 (javulticat FF121: M1 Pro AArch64/ARM64) *

galaxyA30 64bit   21689    6506     30226     9066
      Pixel7Pro    6506    6506      9070     9070 (javulticat FF121: AArch64/ARM64) *

* these are "64bit-only" devices

So what I'm interested in is on any device (os, vm), with any gecko browser (TB, FF) and any version - can you past me the results - just tell me those parameters - browser bitness, os, vm + underlying os, browser (TB, FF), version ... and tell me if you have opened it the browser with javascript.options.baselinejit = true or false. If you want, test both true and false (i.e change the value and restart)

TIA

rusty-snake commented 9 months ago

Fedora 39; x86_64; Firefox 121 (from Fedora); baselinejit,ion=false;

Thorin-Oakenpants commented 9 months ago

i'm updating OP with results as I go

therealmate commented 9 months ago

Arch Linux; 64 bit; Firefox 121.0-1; javascript.options.baselinejit=false

javascript.options.baselinejit=true:

iam-py-test commented 9 months ago

Firefox 121.0 (64-bit) running on Windows 10 22H2 (32.0 GB of RAM). Firefox is running on default settings (javascript.options.baselinejit = true) With javascript.options.baselinejit=false:

level: 11788 stack: 9344

Thanks

Thorin-Oakenpants commented 9 months ago

@iam-py-test I don't think your worker result for baselinejit = true is correct

iam-py-test commented 9 months ago

Sorry Re-did and got

level: 21572
stack: 8960

Wonder why it didn't work the first time...

Thorin-Oakenpants commented 9 months ago

it worked the first time, my test page is a bit shit to copy paste - i,e you can't do it in one hit - so its just a copy/past human error

therealmate commented 9 months ago

Added javascript.options.baselinejit=true to my post

Thorin-Oakenpants commented 9 months ago

so far this is promising ... android, mac intel, arm (IDK if I'm using the right words! 😁 ) might tell a different story - so far it's looking like equivalency of items that can't really be masked - so the only issue here is likely to be that the pref state on startup can be detected as a bit of information which would add entropy when the user changes the security slider to be the opposite during a session (but I have a cunning plan to fix that)

g-2-s commented 9 months ago

WORKER level: 21551 stack: 8960

DOCUMENT level: 39358 stack: 8448

IFRAME level: 39367 stack: 9344

LMDE6, x64, Firefox 121 mint-001 -1.0, javascript.options.baselinejit = true

WORKER level: 6465 stack: 8960

DOCUMENT level: 11807 stack: 8448

IFRAME level: 11810 stack: 9344

LMDE6, x64, Firefox 121 mint-001 -1.0, javascript.options.baselinejit = false

LMDE6 is basically Debian 12 Bookworm and I see you already got results for that but I still wanted to add to this. Oh and Happy New Year!

EDIT - Here's TB as well if you want:

WORKER level: 6482 stack: 8960

DOCUMENT level: 11816 stack: 8448

IFRAME level: 11814 stack: 9344

LMDE6, x64, Tor Browser 13.0.8, javascript.options.baselinejit = false

WORKER level: 6482 stack: 8960

DOCUMENT level: 11816 stack: 8448

IFRAME level: 11814 stack: 9344

LMDE6, x64, Tor Browser 13.0.8, javascript.options.baselinejit = true

javulticat commented 9 months ago

System: MacBook Pro (16-inch, 2021)

Results: https://arkenfox.github.io/TZP/tests/recursion.html

System: Google Pixel 7 Pro

Thorin-Oakenpants commented 9 months ago

@g-2-s re LMDE6, x64, Tor Browser 13.0.8, javascript.options.baselinejit = true result - that looks like false, not true

Thorin-Oakenpants commented 9 months ago

@javulticat those are great, thanks, except the results have true = false. You need to restart firefox in between tests (on android I use menu>quit - I didn't need to force quit from apps)

g-2-s commented 9 months ago

@g-2-s re LMDE6, x64, Tor Browser 13.0.8, javascript.options.baselinejit = true result - that looks like false, not true

I ran the test again to make sure, still the same result, pic for proof: Screenshot from 2024-01-02 08-10-38 Security Level: Safer

javulticat commented 9 months ago

@javulticat those are great, thanks, except the results have true = false. You need to restart firefox in between tests (on android I use menu>quit - I didn't need to force quit from apps)

@Thorin-Oakenpants I triple-checked my results after changing the config option using a variety of ways to exit and restart the browser (quit normally, force quit and clear cache, restart OS - each time confirming in about:config upon reopening the browser that the setting value remained changed) on both systems since the test results kept coming back identically, but so far I have been unable to get any other set of results on either system, regardless of how javascript.options.baselinejit is set and how the app/system is restarted.

Perhaps this behavior is somehow specific to AArch64 architectures, since I am seeing the same thing on two systems based on that architecture family? Or perhaps my config is enabling/disabling other settings that are causing any change to javascript.options.baselinejit to not have an impact?

Another thing that may be rather unique about my systems is that macOS on Apple silicon is 64-bit-only (from CPU to OS to apps) and the Pixel 7/7 Pro were the first ever 64-bit-only Android phones (likewise, from CPU to OS to apps), despite the fact that 32-bit instruction sets can be enabled on AArch64 chips (and are enabled on almost every other mobile SoC - AFAIK, the Google Fold and Pixel 7a/8/8 Pro, all also based on the Google Tensor G2/G3 SoC, are the only other 64-bit-only Android phones on the market so far).

The only other entry in your list that looks like it may also be from a 64-bit-only AArch64-based system is the M1 iMac (they still make iMacs?), but its results look rather suspect to me, as its doc/iframe:safer result is exactly equal to its worker:standard result, which doesn't occur on any other system on your list, and, at 21690, it's ~10k/2x larger than any other doc/iframe:safer result on your list as well.

I will try the test on my x86_64 Windows 11 machine tomorrow to see if I can get different results on a different architecture, which should help rule out (or confirm 🙃) user error. I can also post my user.js (or values for any specific settings) if that would be helpful.

Thorin-Oakenpants commented 9 months ago

@g-2-s hmmm .. and the slider was at safer when you opened TB? hmmm there might be something else at play here - but this is weird since your FF test didn't exhibit this behavior

@javulticat thanks - this is why we test :)

what gets me is the result for all of them is identical for all six measurements, not even out by 1 - in all three = unlikely IMO - and the TB one changes all the prefs (see below)

There are other prefs that the slider changes - see these - so any of those javascript prefs might also be affecting results.

I only tested the baselinejit one (in FF) as a quick mimic of the slider in TB

Thorin-Oakenpants commented 9 months ago

going to ping @PIeroV for this AArch64 conundrum (IANAHardwareE)

Thorin-Oakenpants commented 9 months ago

the iMac is my desktop - in About>system settings>general .. it says the chip is M1 - it's a year old and is one of these - https://www.apple.com/imac/specs/ (but not from the USA) - I got the lower end spec model in the shop (I had two options) - I only bought it so I could test stuff :)

I'm not super savvy on hardware, but would be nice to know what causes these differences. I initially suspected was hoping that the 21690 jit=off size was a normal thing for macs (not mac intel - I might be getting confused here with terms)

edit: the 21690 was my human error in typing between computer screens or off notes on paper - it is actually 11820 which makes more sense

g-2-s commented 9 months ago

@g-2-s hmmm .. and the slider was at safer when you opened TB? hmmm there might be something else at play here - but this is weird since your FF test didn't exhibit this behavior

Have Standard and Safest as well:

Security Level: Standard, javascript.options.baselinejit = true (it was default true when setting the slider to Standard) WORKER level: 21607 stack: 8960

DOCUMENT level: 39386 stack: 8448

IFRAME level: 39381 stack: 9344

Security Level: Standard, javascript.options.baselinejit = false WORKER level: 6482 stack: 8960

DOCUMENT level: 11816 stack: 8448

IFRAME level: 11814 stack: 9344

Security Level: Safest, javascript.options.baselinejit = false (it was already set to false when setting the slider to Safest, just like with Safer but now to run the test I had to Temp. TRUST arkenfox.github.io via NoScript) WORKER level: 6482 stack: 8960

DOCUMENT level: 11816 stack: 8448

IFRAME level: 11814 stack: 9344

Security Level: Safest, javascript.options.baselinejit = true (to run the test I still had to Temp. TRUST arkenfox.github.io via NoScript) WORKER level: 6482 stack: 8960

DOCUMENT level: 11816 stack: 8448

IFRAME level: 11814 stack: 9344

Thorin-Oakenpants commented 9 months ago

ok, that confirms what I thought. Sorry my test is a pain to copy the results and STR are a bit confusing

original TB - https://github.com/arkenfox/user.js/issues/1789#issuecomment-1873444943

WORKER level: 6482 | DOCUMENT level: 11814 | javascript.options.baselinejit = false
WORKER level: 6482 | DOCUMENT level: 11816 | javascript.options.baselinejit = true
LMDE6, x64, Tor Browser 13.0.8

TB post above

WORKER level: 21607 | DOCUMENT level: 39386 | standard (baselinejit = true)
WORKER level:  6482 | DOCUMENT level: 11816 | standard (baselinejit = false)
   ^ i think this is safer, not standard

i.e you opened at standard + tested, then set to safer + restarted + tested, then set to safest + restarted + tested

so the original post must be another case of copy/paste gone wrong or human confusion? has to be since the new results are consistent with your FF results on the same machine

the two safest tests when you allow JS to run will just return whatever state the browser was started in (for our test on recursion levels and stack lengths)

thanks for testing :) have some 🍕 and 🍰

Thorin-Oakenpants commented 9 months ago

ok, I fubared my iMac result, I just retested - that 21690 - NFI why I wrote that down (not a copy paste job, manually typing between two computer screens)

                     worker         doc/iframe
                standard  safer   standard   safer

     iMac 64bit   21690    6507     39398    21690 (Apple M1)  <-- wrong, human fubar

     iMac 64bit   21690    6507     39398    11820 (Apple M1) <-- correct
2glops commented 9 months ago

FF 121.0 on Debian Bookworm

WORKER level: 21551 stack: 8960

DOCUMENT level: 39358 stack: 8448

IFRAME level: 39367 stack: 9344

javascript.options.baselinejit = true javascript.options.ion = true

jakob11git commented 9 months ago

Firefox 121.0, aarch64 Fedora Linux, M2 MacBook Air

javascript.options.baselinejit = true

WORKER level: 21567 stack: 8960

DOCUMENT level: 39466 stack: 8448

IFRAME level: 39467 stack: 9344

baselinejit_true

javascript.options.baselinejit = false

WORKER level: 6470 stack: 8960

DOCUMENT level: 11840 stack: 8448

IFRAME level: 11840 stack: 9344

baselinejit_false

2glops commented 9 months ago

FF 121.0 Debian Bookworm

javascript.options.baselinejit = false javascript.options.ion = false

WORKER level: 6465 stack: 8960

DOCUMENT level: 11807 stack: 8448

IFRAME level: 11810 stack: 9344

PieroV commented 8 months ago

going to ping @PieroV for this AArch64 conundrum (IANAHardwareE)

Sorry, I definitely don't have enough knowledge about this as well.

But I wouldn't be too surprised if aarch64 was very different from the rest.

TBB 13.0.8, Debian testing:

Standard:

WORKER
level: 21607
stack: 8960

DOCUMENT 
level: 39386
stack: 8448

IFRAME
level: 39381
stack: 9344

Safer:

WORKER 
level: 6482
stack: 8960

DOCUMENT
level: 11816
stack: 8448

IFRAME
level: 11814
stack: 9344

I can test on the same machine on Windows 10 bare metal and on other systems if you give me some more time, let me know :slightly_smiling_face: .

Are you interested on VMs as well? If so I can test Windows 10/11 on a VM and also on Linux VMs.

Thorin-Oakenpants commented 8 months ago

there's already a HW diff in audio - x86/amd vs ARM (IDK what that means)

And here we have some aarch64 being different ( IANAE .. something about 64bit only ) - depending on our step a) results upstream, it may be we always load at disabled and that would solve that (if you know what I'm talking about)

would be nice if someone like ted campbell (I think) from moz weighed in

IDK if VMs are worth it just yet - they seem to mirror the host

Thorin-Oakenpants commented 8 months ago

holy crap that's identical to my VM .. and you said you had different results :P

 debian12 64bit   21607    6482     39386    11816 (VM on windows)
         debian   21607    6482     39386    11816 (pierov)
Thorin-Oakenpants commented 8 months ago

ok, got all I need - thanks everyone