arkenfox / TZP

https://arkenfox.github.io/TZP
MIT License
7 stars 0 forks source link

code help #11

Open Thorin-Oakenpants opened 4 years ago

Thorin-Oakenpants commented 4 years ago

rather than pollute other issues: i'll just use this generic one for help with making code cleaner, or making things work

abrahamjuliot commented 4 years ago

item help code: 2

In the above case, you could use collection instead of collection.map(function(fn) { return fn() }) or a 2d array containing the function reference and function arguments.

let collection = [
    [get_A, []],
    [get_B, ["s"]],
    [get_B, ["n"]],
    // etc
]

Promise.all([
    collection.map(function(functionParts) {
        const functionReference = functionParts[0]
        const functionArguments = functionParts[1]
        return functionReference(...functionArguments)
    })
]).then(function(result) {
    collection.forEach(function(currentValue, index) {
        section.push(result[index])
    })
    // do stuff
})
abrahamjuliot commented 4 years ago

item help code: 1

get_computed_styles currently returns void. It just needs to return a promise that resolves a response. You could return a global promise that resolves in then or return promise.all and return a result in then.

return a promise

function get_computed_styles() {
    // wrap everything in this promise
    return new Promise(resolve => {

        let styleVersion = type => {
            return new Promise(resolve => {
                //...
            })
        }

        Promise.all([
            styleVersion(0),
            styleVersion(1),
            styleVersion(2)
        ]).then(res => {
            // update the dom, etc.

            return resolve(res) // resolve the promise here with res or anything
        }).catch(error => {
            console.error(error)
        })
    })
}

return promise.all

function get_computed_styles() {
    let styleVersion = type => {
        return new Promise(resolve => {
            // ...
        })
    }
    return Promise.all([
        styleVersion(0),
        styleVersion(1),
        styleVersion(2)
    ]).then(res => {
        // update the dom, etc.

        return res // return res or anything here
    }).catch(error => {
        console.error(error)
    })
}
abrahamjuliot commented 4 years ago

item 3

I might be misunderstanding the question, but to dynamically access the array chk3 where n in chk+n is 3, you can place the property in an object (obj.chk3) and then obj['chk'+n] will return the same value as obj.chk3. Here are 2 examples.

object with numbered properties

This line should be removed: let chk0 = [], chk1 = [], chk2 = [], chk3 = []

let obj = {
    chk0: [ ], // obj.chk0 replaces chk0
    chk1: [ ], // obj.chk1 replaces chk1
    chk2: [ ], // obj.chk2 replaces chk2
    chk3: [ ]  // obj.chk3 replaces chk3
}

// when i == 3, obj[`chk${i}`] or obj['chk'+i] will return the array obj.chk3
let compare = obj[`chk${i}`] // no if else check needed

object with numeric properties

let chk = {
    0: [ ], // chk[0] replaces chk0
    1: [ ], // chk[1] replaces chk1
    2: [ ], // chk[2] replaces chk2
    3: [ ]  // chk[3] replaces chk3
}

// when i == 3, chk[i] will return the array chk[3]
let compare = chk[i] // no if else check needed
abrahamjuliot commented 4 years ago

item 2

Since result is a list containing all the responses, you can iterate over it.

Promise.all([
    get_A(),
    get_B("s"),
    get_B("n"),
    // etc
]).then(function(results) {
    results.forEach(function(currentResult) {
        section.push(currentResult)
    })
    // do stuff
})
abrahamjuliot commented 4 years ago

item 4

https://stackoverflow.com/a/3764557 - this is an interesiting post touching on the subject of how sort is not language-sensitive.

abrahamjuliot commented 4 years ago

6b10608 :)

kkapsner commented 4 years ago

Sorry for coming back so late (spare time was rare in the last weeks) - @Thorin-Oakenpants: on what do you still need help?

kkapsner commented 4 years ago

So the idea is

Sounds like a good idea.

I'm using three worker js files, three lots of code to check for them, etc. The CB test only uses one worker js file.

The three different worker types are managed by https://github.com/kkapsner/CanvasBlocker/blob/master/test/navigatorTestWorker.js#L17-L42

https://github.com/kkapsner/CanvasBlocker/blob/master/test/navigatorTestWorker.js#L44-53 deals with the nested workers.

he other thing I was thinking, was using a global worker to return all results: all the language stuff, all the navigator properties etc. These results would go into a global array which each section can then just look up what they need. This way I only need to run a single test on page load, or when I rerun a section or rerun all, I clear the global array and do it again

I would delay such performance optimizations. It would make the code harder to code/read/maintain. Starting with small independent sections is way easier to develop.

Maybe I'm better off with a single worker file per section

Yes - I think so.

abrahamjuliot commented 4 years ago

Enumerating elements or window objects has some gains in singling out (and guessing) versions and/or user triggered changes. I also target certain window objects and perform tampering tests on near every property per prototype.

If a native preference or form of tampering triggers a change to HTMLAnchorElement or HTMLLinkElement, then it might be worth enumerating and checking.

Object.keys(HTMLAnchorElement.prototype)
Object.getOwnPropertyNames(HTMLAnchorElement.prototype) // includes the constructor
abrahamjuliot commented 4 years ago

ping, charset, hreflang, referrerPolicy

I'm not sure on this. I have not researched or tested it. There might be some unique behavior if we test a link with these attributes.

abrahamjuliot commented 4 years ago

item 7

8551b01 ✔

abrahamjuliot commented 3 years ago

Yes, that's it.

abrahamjuliot commented 3 years ago

This will get each property/value

const anchorElement = document.createElement('a') // or a live tag: document.getElementById('test-anchor')
const propertyNames = Object.getOwnPropertyNames(anchorElement.__proto__)
const obj = {}
propertyNames.forEach(propName => {
    const value = anchorElement[propName]
    return (
        obj[propName] = value
    )
})

console.log(obj)
abrahamjuliot commented 3 years ago

Here's how I would style the modal overlay and content.

/* parent overlay */
.modal-overlay {
  position: fixed;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
  overflow: auto; /* scroll when content overflows */
  visibility: hidden; /* toggle to show the modal (optionally use display instead)*/
}

/* child cotent */
.modal-content {
  max-width: calc(900px * 0.9); /* 90% of 900px */
  position: absolute;
  top: 0;
  left: 0;
  right: 0;
}

I use a modified pure CSS approach inspired by http://youmightnotneedjs.com/#modal from https://youmightnotneed.com.

<div>
  <!-- Modal Open -->
  <input id="open-modal-1" name="modal-1" type="radio">
  <label for="open-modal-1" onclick="">[open modal-1]</label>
  <!-- Modal Overlay/Modal Close -->
  <label for="close-modal-1" class="modal-overlay" onclick="">
    <!-- Modal Content -->
    <label for="open-modal-1" class="modal-content" onclick="">
      <!-- Close -->
      <input id="close-modal-1" name="modal-1" type="radio">
      <label for="close-modal-1" class="close-btn" onclick="">×</label>
      <!-- Content -->
      <div>modal-1 Content</div>
    </label>
  </label>
</div>

<!--
- modal-1 can be something unique like modal-canvas 
- add more modals by changing all occurances of "modal-1" to modal-a-unique-name in the html and css
-->
input[type="radio"][name^="modal"] {
  display: none;
}
.modal-overlay,
[for^="open-modal"],
[for^="close-modal"] {
  cursor: pointer;
}
.modal-overlay {
  background: rgba(0, 0, 0, 0.9);
  position: fixed;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
  visibility: hidden;
  overflow: auto;
  z-index: 1000;
}
.modal-content {
  background: #fff;
  border: 1px solid;
  margin: 100px auto;
  padding: 20px 30px;
  max-width: calc(400px * 0.9);
  position: absolute;
  top: 0;
  left: 0;
  right: 0;
  cursor: text;
}
.close-btn {
  position: absolute;
  top: 0;
  right: 8px;
  font-size: 30px;
}

/* modal-1 component */
[name="modal-1"]:checked ~ .modal-overlay {
  visibility: visible;
}
[name="modal-1"]:not(:checked) ~ .modal-overlay {
  visibility: hidden;
}

live example: https://jsfiddle.net/oap06rf1/

Thorin-Oakenpants commented 3 years ago

now this is what I'm talking about

Not sure if I should go pure css, because I need the text to trigger js anyway (to populate the overlay content)

abrahamjuliot commented 3 years ago

modal content scrollbar

To include a scroll bar in the modal content when the max height is reached, you can set max-height: 90% and overflow-y: auto. Here's style similar to the middle screen overlay: https://jsfiddle.net/1bu2xy3a.

`

abrahamjuliot commented 3 years ago

Looking good.

Here's a regex short circuit version of the test, but if blocks read better in my opinion.

function check_basics(str, property) {
    const dynamicReturns = /(undefined (value|string)|blocked|empty string|(^\s{1}|\s{1}$|\s{2,}))/
    const firefox = /\srv:(\d|\.)+\) Gecko\/(20100101|(\d|\.)+) Firefox\//
    const webkit = /webkit/i  // or /gecko\/.+webkit|webkit.+gecko\//i
    const bs = (
        dynamicReturns.test(str) ||
        (property == 'userAgent' ? !firefox.test(str) || webkit.test(str) : false)
    )
    return bs
}
Thorin-Oakenpants commented 3 years ago

I think regex might be overkill, but I kinda like reducing the lines: however, regex is too confusing for me, and I have to work with the code, so probably better I don't use it

Thorin-Oakenpants commented 3 years ago

@abrahamjuliot one thing I noticed was that Brave (not chrome or opera) document/iframes sometimes add a double trailing space to userAgent and appVersion - I emailed @pes10k

weird

abrahamjuliot commented 3 years ago

FxiOS

Correct, it's on Webkit and should fail the Gecko test.

Brave

https://github.com/brave/brave-browser/issues/9190 - document/iframes (+web workers in nightly)

double space already present in longer spaces

That is short and sweet. My thinking is upside down on that one. 🤦‍♂️🙃

Thorin-Oakenpants commented 3 years ago

^^ ahh, right ... so Brave doesn't protect workers then .. sheesh

And in fact, you could use if (isBrave) {remove all leading, trailing and double spaces until no more double spaces exist} .. bam, randomizing stopped and real value exposed edit: correction: the randomizing is probably neutralized (depends on how far one wants to check what is being done and how to counter it)

pes10k commented 3 years ago

Just to clarify, the above is not entirely correct.

  1. There are fingerprinting protection changes that are applied everywhere (removing device info on android devices)
  2. There is a long list of things in blink that are tricky to do in workers. This is one of them (the canvas item you identified before is a second)

If it saves ya'll time to, here are the manual tests we use for checking farbling protections in a number of cases, if it helps

https://dev-pages.brave.software/farbling.html

pes10k commented 3 years ago

Will also note we're tracking the UA in workers issue here https://github.com/brave/brave-browser/issues/12392

Thorin-Oakenpants commented 3 years ago

Thanks @pes10k .. patch those holes man :) I'm adding code here to show that extensions lack APIs to do the job, and that FPing generally needs to be applied internally - it keeps throwing me when I see results like this in Brave (I very occasionally flick open chromium browsers to see what happens)

edit: corrected my statement two posts up: I work on FF stuff, so I'm not particularly looking at Brave when I want to unmask randomizing (to return a static value for all users in that browser) or find a bug/other-method (to get entropy more than just a static value).

PPS: thanks for the brave dev test page

pes10k commented 3 years ago

@Thorin-Oakenpants i think thats a known issue; FF doesn't seem too support the API (https://caniuse.com/offscreencanvas). Do you know if they have some other way of doing worker-side canvas operations?

This snowballing test suite is something i maintain for the QA folks at Brave. Im happy to make a change or two if it'd be helpful for ya'll, but would just need specific asks and pointers :)

Thorin-Oakenpants commented 3 years ago

No need to make changes just for me - just seems weird that it error'd: i mean even if offscreen canvas in FF had issues (I enabled it: the code is there behind a pref), you'd think that the other worker checks would still pass. But to be fair, it is a Brave test page :)

Thorin-Oakenpants commented 2 years ago

@abrahamjuliot I'm trying to color up differences in two strings

example

Wednesday, January 30, 2019, 1:00:00 PM Coordinated Universal Time
Wednesday, 30 January 2019, 1:00:00 pm Coordinated Universal Time

result e.g. bold parts would be red

really struggling to find something - any ideas?

Thorin-Oakenpants commented 2 years ago

^ only if this is easy to drop in, otherwise it's not worth it. Need to handle RTL etc and not drop any chars but pick up any char diffs

PS: work in progress: https://arkenfox.github.io/TZP/tests/formatting.html

But the rest looks solid: i.e the preferred language should return the expected formatting. FF unfortunately uses Region, not "app"

abrahamjuliot commented 2 years ago

Here's my shot at this. It should handle line & word diffs fine.

dateString1 = 'Wednesday, January 30, 2019, 1:00:00 PM Coordinated Universal Time'
dateString2 = 'Wednesday, 30 January 2019, 1:00:00 pm Coordinated Universal Time'

getDiffs = ({ stringA, stringB, charDiff = false, decorate = diff => `[${diff}]` }) => {
    const splitter = charDiff ? '' : ' '
    const listA = (''+stringA).split(splitter)
    const listB = (''+stringB).split(splitter)
    const listBWithDiffs = listB.map((x, i) => {
        const matcher = listA[i]
        const match = x == matcher
        return !match ? decorate(x) : x
    })
    return listBWithDiffs.join(splitter)
}

// example with custom options

getDiffs({
    stringA: dateString1,
    stringB: dateString2,
    //charDiff: true // use this option for single words, hashes or number
    decorate: diff => `<span style="color: red">${diff}</span>` // custom function with css style
})

// default example

getDiffs({ stringA: dateString2, stringB: dateString1 })
//=> 'Wednesday, [January] [30,] 2019, 1:00:00 [PM] Coordinated Universal Time'

getDiffs({ stringA: dateString1, stringB: dateString2 })
//=> 'Wednesday, [30] [January] 2019, 1:00:00 [pm] Coordinated Universal Time'
Thorin-Oakenpants commented 2 years ago

spiffy .. I'll add it later on, thanks Satan Santa

Thorin-Oakenpants commented 2 years ago

anyway to make this a friendly function for me?

function colorDiff ( stringA, stringB, decOpen, decClose = "sc", charDiff = false ) {
   // returns the second string with diffs colored compared to first string
   // ^ IDK about you but that should be the other way round

}

let coloredStringExpected = colorDiff(strGot, strExpected, "sb")

anyway this is a bit whack - the second one is fubar, did I do something wrong?

const color_diffs = ({ stringA, stringB, charDiff = false, decorate = diff => `[${diff}]` }) => {
    const splitter = charDiff ? '' : ' '
    const listA = (''+stringA).split(splitter)
    const listB = (''+stringB).split(splitter)
    const listBWithDiffs = listB.map((x, i) => {
        const matcher = listA[i]
        const match = x == matcher
        return !match ? decorate(x) : x
    })
    return listBWithDiffs.join(splitter)
}

function whatever() {
    for (let i=0; i < arrayS.length; i++) {
        // expected vs got
        let strE = arrayS[i], strG = arrayU[i]
        if (strE !== strG) {
            strE = color_diffs({
                    stringA: strG,
                    stringB: strE,
                    //charDiff: true // use this option for single words, hashes or number
                    decorate: diff => sb + `${diff}` + sc
            })
            strG = color_diffs({
                    stringA: strE,
                    stringB: strG,
                    decorate: diff => sb + `${diff}` + sc
            })
            diffs.push(s12 + aIndex[i] + sc)
            let str = "<ul><li>"+ strE +"</li><li>"+ strG +"</li></ul>"
            diffs.push(str)
        }
    }
} 

whack

Thorin-Oakenpants commented 2 years ago

oh snap, I modified strE and then reused it 🤦‍♀️ sorry Santa

Thorin-Oakenpants commented 2 years ago

still something a little off - see the Int;.DateTimeFormat lines whack

abrahamjuliot commented 2 years ago

This cleans it up a bit. I added a search in reverse order.

original = "1/30/2019, 1:00 PM Jan 30, 2019, 1:00:00 PM January 30, 2019 at 1:00:00 PM UTC"
changed = "30/01/2019, 1:00 pm 30/01/2019, 1:00:00 pm 30 January 2019 at 1:00:00 pm UTC"

getDiffs = ({ stringA, stringB, charDiff = false, decorate = diff => '['+diff+']' }) => {
    const splitter = charDiff ? '' : ' '
    const listA = (''+stringA).split(splitter)
    const listB = (''+stringB).split(splitter)
    const diffIndexList = []
    listB.forEach((x, i) => {
        const matcher = listA[i]
        const match = x == matcher
        if (!match) {
            diffIndexList.push(i)
        }
        return
    })
    // 🎅🏻
    const listAReversed = [...listA].reverse()
    const listBWithDiffs = [...listB].reverse().map((x, i) => {
        const matcher = listAReversed[i]
        const match = x == matcher
        const index = listB.length-(i+1)
        if (diffIndexList.includes(index)) {
            if (match) {
                diffIndexList.splice(diffIndexList.indexOf(index), 1)
                return x
            }
            return decorate(x)
        }
        return x
    })
    return listBWithDiffs.reverse().join(splitter)
}

getDiffs({ stringA: original, stringB: changed })
//=> "[30/01/2019,] 1:00 [pm] [30/01/2019,] 1:00:00 [pm] [30] [January] 2019 at 1:00:00 [pm] UTC"

getDiffs({ stringA: changed, stringB: original })
//=> "[1/30/2019,] 1:00 [PM] [Jan] [30,] [2019,] 1:00:00 [PM] [January] [30,] 2019 at 1:00:00 [PM] UTC"
Thorin-Oakenpants commented 2 years ago

much spiffier and good enough .. will be interesting to see how it handled non-western and LTR

Thorin-Oakenpants commented 2 years ago

@abrahamjuliot I got two things for you

ONE

Where, how is are those two Type Errors being thrown? I'd like to trap it for entropy


TWO

abrahamjuliot commented 2 years ago

If I reload with the console open, the errors should log with function line detail. Looks like the error is trigger in these 2 lines when we access contentWindow.

https://github.com/arkenfox/TZP/blob/7039ec73d16aa0bb47c521a995f518348550c528/js/misc.js#L73

https://github.com/arkenfox/TZP/blob/7039ec73d16aa0bb47c521a995f518348550c528/js/generic.js#L918

Looks like this is the tampering code line. 'get contentWindow' should not exist on the prototype.

HTMLIFrameElement.prototype['get contentWindow']
abrahamjuliot commented 2 years ago

I'll test the audio in a VM and see what I can find. Sounds like memory leak in the extension.

Thorin-Oakenpants commented 2 years ago

I can't replicate today (version hasn't changed since Nov last year) hmmm

edit: @abrahamjuliot don't bother testing unless you really want to - I think it's intermittent

Thorin-Oakenpants commented 2 years ago
//from globals
let zU = "undefined",
    zUQ = "\"undefined\""

function cleanFn(item) {
    // catch strings as strings, tidy undefined, empty strings
    if (typeof item == "number" || typeof item == "bigint") { return item
    } else if (item == zU) {item = zUQ
    } else if (item == "true") {item = "\"true\""
    } else if (item == "false") {item = "\"false\""
    } else if (item == "null") {item = "\"null\""
    } else if (!skipArray && Array.isArray(item)) {
        item = !item.length ? "empty array" : "array"
    } else if (item === undefined || item === true || item === false || item === null) {item += ""
    } else if (item == "") {item = "empty string"
    } else if (typeof item == "string") {
        if (!isNaN(item*1)) {item = "\"" + item + "\""}
    }
    return item
}

@abrahamjuliot - how can I catch an empty array

abrahamjuliot commented 2 years ago

This should do the trick

a = []
isEmptyArray = a => Array.isArray(a) && !a.length
isEmptyArray(a) // true
Thorin-Oakenpants commented 2 years ago

I must be tired, I tried Array,isArray etc .. I had put it after checking for "" 🤦 - function edited above

Thorin-Oakenpants commented 2 years ago

nifty: https://developer.chrome.com/blog/what-is-the-top-layer/

Thorin-Oakenpants commented 1 year ago

@abrahamjuliot .. in looking for paper cuts, I've found (e.g. for larger arrays such as my font list building), using .push.apply is faster than concat - any downsides to this? IIUIC this saves recreating the first array - right?

e.g. a1 = a1.concat(a2) vs a1.push.apply(a1, a2)

abrahamjuliot commented 1 year ago

Good tip. I have not looked into it, but it makes sense. concat creates a new array per iteration, and push modifies the existing array. There are no downsides unless we want to preserve the original a1 array. We could always map a copy of it.

Thorin-Oakenpants commented 1 year ago

I did a bunch of tests (using the font lists from TZP) where for the current OS it creates, one time, a full list and a base (kBaseFonts, whitelist - depending on if TB or not) list

do 12 tests on each, wait a second or two between tests, remove highest + lowest outliers, average

windows FF

testmerge("windows", false, 1) let array = [0.5118186958134174, 0.3838640218600631, 0.41323485458269715, 0.35500398511067033, 0.36802931129932404, 0.5169266662560403, 0.3777344562113285, 0.5496176811866462, 0.41783202951774, 0.3787560509517789] 10 totalling 3.7609990569762886 testmerge("windows", false, 2) [0.2252615219913423, 0.23215728206560016, 0.23522206535562873, 0.26229431154206395, 0.26689148554578424, 0.2845139857381582, 0.23343427572399378, 0.21249159425497055, 0.22807090589776635, 0.21938735526055098] 10 totalling 2.174463261384517

old .376 vs new .217 - save .16


mac
```js

testmerge("mac", false, 1)
[0.6969826449640095, 0.5128402896225452, 0.509264709893614, 0.484235652256757, 0.5363369565457106, 0.48704503616318107, 0.4737643115222454, 0.4924084055237472, 0.5228008339181542, 0.48857742780819535]
10 totalling 4.50727362325415
testmerge("mac", false, 2)
[0.3350828983820975, 0.3767128624022007, 0.465336159337312, 0.3703278978355229, 0.4382639126852155, 0.3611335502937436, 0.34887441946193576, 0.3593457601964474, 0.39944333396852016, 0.41272405721247196]
10 totalling 3.53216195339337

old .451 vs new .353 = save .098

of course this is on an old PC (had it's 11th birthday two weeks ago), and macs should be pretty grunty. So who knows, maybe 0.05 ms on a page load (fntLists are only set once)

Anyway, I found a massive perf boost by not writing to the DOM until the end of all the FP data collection (except screen which is real-time on resize event) - ~20ms .. so take that

At the moment the live master (as a local copy) and my refactoring (sooooo many changes - AND additional stuff) both run as file:// and loaded several times to remove any cold starts/latency are approx 310ms master and 235ms (refactor)

master live on my mac as from memory 102ms .. not sure how much more perf I can wring out of this... only to then add new tests (webgl, font variants, etc)

anyway .. happy fat-red-fucker day ... going to go give mrs 🤶 a stuffing 👋

Thorin-Oakenpants commented 1 year ago

@abrahamjuliot : is there any real benefit to using promises vs functions : i.e return vs return resolve? I thought I saw some PRs in creepy where you removed some for perf?

e.g. - except get_fonts (which uses get_font_sizes) and get_unicode - all are simple and super fast: e.g. out of say 120ms to run the entire fonts section, 117ms is get_fonts and get_unicode

promise-vs-function

abrahamjuliot commented 1 year ago

promises vs functions

If there are no event-based operations in the function like fetch, timeouts, listeners, or built in promise-based APIs, I don't see any major benefit to making functions promises, except maybe to try/catch errors with different syntax. In any case, we can place both promise and non-promise functions in a Promise.all or Promise.allSettled and then use Promise.catch.

However, there can be some performance advantages in making functions event-based by using setTimeout(fn, 0). Interesting thread on the subject. That's what I experimented with, but I might tone it down in some areas. It's tricky to find a sweet spot on performance when there's already a lot going on in the event loop. I'm trying to wrap my mind around some of these concepts here.

Thorin-Oakenpants commented 1 year ago

there can be some performance advantages in making functions event-based

indeed, I found by doing this - but it was years ago, that it at least gives me more accurate perf results per section

You can also see it here .. and that was only six months ago

IDK if it improves overall perf. I could test. I do know that when RFP is on, I need to run devices very early or it holds everything up - I need to retest that with my refactored version

At the moment the only things that affect timing are IDK what the word, things like media devices (promised), fonts, canvas (promised). I need to learn more about async and stuff - that link looks like a good read

Thorin-Oakenpants commented 1 year ago

hey @abrahamjuliot , how do you handle undefined in your JSONs etc. I kinda have two issues

display

I wonder if the display side of things (an empty string aside) is even worth it - the underlying data/hash knows the difference. Everything is parsed thru a display function, so I could split the passed data into value and notation and modify as required

however, undefined will not show up in JSON

I was thinking of trying to catch undefined everywhere and recording it as typeof undefined, but this is problematic and somewhat messy (but not impossible)

is undefined really an issue? If it's not in the JSON, then that's why (or it wasn't a collected metric at the time) - e.g. import JSON into database etc. And a user clicks [10 metrics] and only sees 9 metrics then that's why.

Just wondering what you do?