phetsims / qa

Quality Assurance Task Tracking
MIT License
13 stars 8 forks source link

Maintenance Spot Check (Analytics) #1063

Closed jonathanolson closed 6 months ago

jonathanolson commented 7 months ago

I've patched all release branches with https://github.com/phetsims/phetcommon/issues/64, https://github.com/phetsims/phetcommon/issues/65, and https://github.com/phetsims/phetcommon/issues/66.

I've done some testing in https://github.com/phetsims/perennial/blob/main/js/scripts/test/release-branch-checks.js (passing for all release branches), but I'd like:

  1. @oliver-phet, can you do whatever spot checks for those three issues we can do (especially the EU-based testing, and the yotta query-parameter forwarding?)
  2. QA spot checks (looking in the network view) to check that https://github.com/phetsims/phetcommon/issues/65 is working (if you provide ?yotta=false, we shouldn't be sending external network requests, e.g. anything that is yotta/google-analytics/etc. should not happen).
  3. QA spot checks that simulations are running ok (my scripts I believe should have ensured that errors aren't happening on sim load, but either brief checks or spot-checks to make sure makes sense).

acid-base-solutions 1.3 (https://github.com/phetsims/phetcommon/issues/66)

area-builder 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

area-model-algebra 1.3 (https://github.com/phetsims/phetcommon/issues/66)

area-model-decimals 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

area-model-introduction 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

area-model-multiplication 1.3 (https://github.com/phetsims/phetcommon/issues/66)

arithmetic 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

atomic-interactions 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

balancing-act 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

balancing-chemical-equations 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

balloons-and-static-electricity 1.3-phetio (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

balloons-and-static-electricity 1.5 (https://github.com/phetsims/phetcommon/issues/66)

beers-law-lab 1.7 (https://github.com/phetsims/phetcommon/issues/66)

bending-light 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

blackbody-spectrum 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

build-a-fraction 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

build-a-molecule 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

build-a-nucleus 1.1 (https://github.com/phetsims/phetcommon/issues/66)

build-an-atom 1.5-phetio (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

build-an-atom 1.6 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

calculus-grapher 1.0 (https://github.com/phetsims/phetcommon/issues/66)

capacitor-lab-basics 1.6 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

center-and-variability 1.1 (https://github.com/phetsims/phetcommon/issues/66)

charges-and-fields 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

charges-and-fields 1.0-phetio (https://github.com/phetsims/phetcommon/issues/66)

circuit-construction-kit-ac 1.0 (https://github.com/phetsims/phetcommon/issues/66)

circuit-construction-kit-ac-virtual-lab 1.0 (https://github.com/phetsims/phetcommon/issues/66)

circuit-construction-kit-dc 1.3 (https://github.com/phetsims/phetcommon/issues/66)

circuit-construction-kit-dc-virtual-lab 1.3 (https://github.com/phetsims/phetcommon/issues/66)

collision-lab 1.1 (https://github.com/phetsims/phetcommon/issues/66)

color-vision 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

color-vision 1.2-phetio (https://github.com/phetsims/phetcommon/issues/66)

concentration 1.5-phetio (https://github.com/phetsims/phetcommon/issues/66)

concentration 1.7 (https://github.com/phetsims/phetcommon/issues/66)

coulombs-law 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

curve-fitting 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

density 1.1 (https://github.com/phetsims/phetcommon/issues/66)

diffusion 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

energy-forms-and-changes 1.4 (https://github.com/phetsims/phetcommon/issues/66)

energy-skate-park 1.3 (https://github.com/phetsims/phetcommon/issues/66)

energy-skate-park-basics 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

energy-skate-park-basics 1.3-phetio (https://github.com/phetsims/phetcommon/issues/66)

equality-explorer 1.1 (https://github.com/phetsims/phetcommon/issues/66)

equality-explorer-basics 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

equality-explorer-two-variables 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

expression-exchange 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

faradays-law 1.3-phetio (https://github.com/phetsims/phetcommon/issues/66)

faradays-law 1.4 (https://github.com/phetsims/phetcommon/issues/66)

forces-and-motion-basics 2.1-phetio (https://github.com/phetsims/phetcommon/issues/66)

forces-and-motion-basics 2.3 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

fourier-making-waves 1.0 (https://github.com/phetsims/phetcommon/issues/66)

fraction-matcher 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

fractions-equality 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

fractions-intro 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

fractions-mixed-numbers 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

friction 1.6 (https://github.com/phetsims/phetcommon/issues/66)

function-builder 1.2 (https://github.com/phetsims/phetcommon/issues/66)

function-builder-basics 1.2 (https://github.com/phetsims/phetcommon/issues/66)

gas-properties 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

gases-intro 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

gene-expression-essentials 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

geometric-optics 1.3 (https://github.com/phetsims/phetcommon/issues/66)

geometric-optics-basics 1.3 (https://github.com/phetsims/phetcommon/issues/66)

graphing-lines 1.3 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

graphing-quadratics 1.3 (https://github.com/phetsims/phetcommon/issues/66)

graphing-slope-intercept 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

gravity-and-orbits 1.5 (https://github.com/phetsims/phetcommon/issues/66)

gravity-and-orbits 1.6 (https://github.com/phetsims/phetcommon/issues/66)

gravity-force-lab 2.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

gravity-force-lab-basics 1.1 (https://github.com/phetsims/phetcommon/issues/66)

greenhouse-effect 1.0 (https://github.com/phetsims/phetcommon/issues/66)

greenhouse-effect 1.2 (https://github.com/phetsims/phetcommon/issues/66)

hookes-law 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

isotopes-and-atomic-mass 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

john-travoltage 1.4-phetio (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

john-travoltage 1.6 (https://github.com/phetsims/phetcommon/issues/66)

keplers-laws 1.2 (https://github.com/phetsims/phetcommon/issues/66)

least-squares-regression 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

make-a-ten 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

masses-and-springs 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

masses-and-springs-basics 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

mean-share-and-balance 1.0 (https://github.com/phetsims/phetcommon/issues/66)

molarity 1.4 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

molarity 1.5 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

molecule-polarity 1.3 (https://github.com/phetsims/phetcommon/issues/66)

molecule-shapes 1.6 (https://github.com/phetsims/phetcommon/issues/66)

molecule-shapes-basics 1.6 (https://github.com/phetsims/phetcommon/issues/66)

molecules-and-light 1.3-phetio (https://github.com/phetsims/phetcommon/issues/66)

molecules-and-light 1.5 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

my-solar-system 1.3 (https://github.com/phetsims/phetcommon/issues/66)

natural-selection 1.2 (https://github.com/phetsims/phetcommon/issues/66)

natural-selection 1.3 (https://github.com/phetsims/phetcommon/issues/66)

natural-selection 1.4 (https://github.com/phetsims/phetcommon/issues/66)

natural-selection 1.5 (https://github.com/phetsims/phetcommon/issues/66)

neuron 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

normal-modes 1.0 (https://github.com/phetsims/phetcommon/issues/66)

number-compare 1.0 (https://github.com/phetsims/phetcommon/issues/66)

number-line-distance 1.1 (https://github.com/phetsims/phetcommon/issues/66)

number-line-integers 1.2 (https://github.com/phetsims/phetcommon/issues/66)

number-line-operations 1.1 (https://github.com/phetsims/phetcommon/issues/66)

number-play 1.1 (https://github.com/phetsims/phetcommon/issues/66)

ohms-law 1.4 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

pendulum-lab 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

ph-scale 1.6 (https://github.com/phetsims/phetcommon/issues/66)

ph-scale-basics 1.5 (https://github.com/phetsims/phetcommon/issues/66)

ph-scale-basics 1.6 (https://github.com/phetsims/phetcommon/issues/66)

plinko-probability 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

projectile-motion 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

proportion-playground 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

quadrilateral 1.1 (https://github.com/phetsims/phetcommon/issues/66)

ratio-and-proportion 1.2 (https://github.com/phetsims/phetcommon/issues/66)

reactants-products-and-leftovers 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

resistance-in-a-wire 1.3-phetio (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

resistance-in-a-wire 1.6 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

rutherford-scattering 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

sound-waves 1.1 (https://github.com/phetsims/phetcommon/issues/66)

states-of-matter 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

states-of-matter-basics 1.2 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

trig-tour 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

under-pressure 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

unit-rates 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

vector-addition 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

vector-addition-equations 1.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

wave-interference 2.0 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

wave-on-a-string 1.1 (https://github.com/phetsims/phetcommon/issues/66, https://github.com/phetsims/phetcommon/issues/65)

waves-intro 1.1 (https://github.com/phetsims/phetcommon/issues/66)

KatieWoe commented 6 months ago

Noting that the on-going issue of Instance Proxies not launching is still an issue.

Nancy-Salpepi commented 6 months ago

Noting that with mac + chrome and with Win12 + chrome the following issue appears in the console with ?yotta=false:

Screenshot 2024-03-26 at 4 02 26 PM
KatieWoe commented 6 months ago

Calling QA here.

oliver-phet commented 6 months ago

@JacquiHayes Can you test running some sims from the EU? https://phet-dev.colorado.edu/html/acid-base-solutions/1.3.5-rc.1/phet/acid-base-solutions_all_phet.html https://phet-dev.colorado.edu/html/area-builder/1.1.33-rc.1/area-builder_all.html https://phet-dev.colorado.edu/html/area-model-algebra/1.3.1-rc.1/phet/area-model-algebra_all_phet.html

We need to verify that these versions are properly disabling Google Analytics cookie storage. If you open your browser console and review the Network tab, there will be several entries for www.google-analytics.com and I'd like to verify that gcs=G100 is listed (this means analyticsStorage is denied).

Here's what it looks like on my machine (gcs=G101 meaning analyticsStorage is granted since I'm in the US).

Screenshot 2024-03-26 at 2 22 04 PM

We can collaborate synchronously on a call if that is easier!

oliver-phet commented 6 months ago

@jacquihayes did some live testing on a call today and verified gcs=G100 was present in all test sims (using Chrome and Safari) and no cookies were saved. This test was done from Austria. However, when connecting via ExpressVPN, gcs=G101 was present. She is going to connect with another colleague physically located in Germany to verify.

oliver-phet commented 6 months ago

Update: Jacqui's contact in the Netherlands also verified gcs=G100 and no cookies.

oliver-phet commented 6 months ago

@jonathanolson on our call today, we realized it may be helpful to add 'CH' (Switzerland) and 'GB' (United Kingdom) to our exception regions. (NOTE: we did some Googling and are pretty sure GB is the region code to use for United Kingdom, could you please verify)

Would this be an easy addition to add to this MR (@kathy-phet and I hope it wouldn't require any QA since it is a minimal change, but would appreciate your comment!)

oliver-phet commented 6 months ago

We are going to wait a day for a contact in Germany to manually verify before next steps.

jonathanolson commented 6 months ago

It shouldn't be too hard to apply those patches, and I don't think it will need additional QA besides my checks. On it.

oliver-phet commented 6 months ago

Jacqui's contact in Hamburg, Germany was able to manually verify denial of analytics storage (gcs=G100)!

jonathanolson commented 6 months ago

https://en.wikipedia.org/wiki/ISO_3166-2#current-codes is the spec for regions, as noted in https://developers.google.com/tag-platform/tag-manager/templates/consent-apis#regional-behavior.

Those regions look correct, I'll add them with patches!

jonathanolson commented 6 months ago

Deployed list:

acid-base-solutions 1.3 ()

area-builder 1.1 ()

area-model-algebra 1.3 ()

area-model-decimals 1.2 ()

area-model-introduction 1.2 ()

area-model-multiplication 1.3 ()

arithmetic 1.0 ()

atomic-interactions 1.2 ()

balancing-act 1.1 ()

balancing-chemical-equations 1.2 ()

balloons-and-static-electricity 1.3-phetio ()

balloons-and-static-electricity 1.5 ()

beers-law-lab 1.7 ()

bending-light 1.1 ()

blackbody-spectrum 1.0 ()

build-a-fraction 1.0 ()

build-a-molecule 1.0 ()

build-a-nucleus 1.1 ()

build-an-atom 1.5-phetio ()

build-an-atom 1.6 ()

calculus-grapher 1.0 ()

capacitor-lab-basics 1.6 ()

charges-and-fields 1.0 ()

charges-and-fields 1.0-phetio ()

circuit-construction-kit-ac-virtual-lab 1.0 ()

circuit-construction-kit-dc 1.3 ()

circuit-construction-kit-dc-virtual-lab 1.3 ()

collision-lab 1.1 ()

color-vision 1.1 ()

color-vision 1.2-phetio ()

concentration 1.5-phetio ()

concentration 1.7 ()

coulombs-law 1.0 ()

curve-fitting 1.0 ()

density 1.1 ()

diffusion 1.0 ()

energy-forms-and-changes 1.4 ()

energy-skate-park 1.3 ()

energy-skate-park-basics 1.1 ()

energy-skate-park-basics 1.3-phetio ()

equality-explorer 1.1 ()

equality-explorer-basics 1.0 ()

equality-explorer-two-variables 1.0 ()

expression-exchange 1.1 ()

faradays-law 1.3-phetio ()

faradays-law 1.4 ()

forces-and-motion-basics 2.1-phetio ()

forces-and-motion-basics 2.3 ()

fourier-making-waves 1.0 ()

fraction-matcher 1.2 ()

fractions-equality 1.1 ()

fractions-intro 1.0 ()

fractions-mixed-numbers 1.0 ()

friction 1.6 ()

function-builder 1.2 ()

function-builder-basics 1.2 ()

gas-properties 1.0 ()

gases-intro 1.0 ()

gene-expression-essentials 1.0 ()

geometric-optics 1.3 ()

geometric-optics-basics 1.3 ()

graphing-lines 1.3 ()

graphing-quadratics 1.3 ()

graphing-slope-intercept 1.1 ()

gravity-and-orbits 1.5 ()

gravity-and-orbits 1.6 ()

gravity-force-lab 2.2 ()

gravity-force-lab-basics 1.1 ()

greenhouse-effect 1.0 ()

greenhouse-effect 1.2 ()

hookes-law 1.0 ()

isotopes-and-atomic-mass 1.1 ()

john-travoltage 1.4-phetio ()

john-travoltage 1.6 ()

keplers-laws 1.2 ()

least-squares-regression 1.1 ()

make-a-ten 1.0 ()

masses-and-springs 1.0 ()

masses-and-springs-basics 1.0 ()

mean-share-and-balance 1.0 ()

molarity 1.4 ()

molarity 1.5 ()

molecule-polarity 1.3 ()

molecule-shapes 1.6 ()

molecule-shapes-basics 1.6 ()

molecules-and-light 1.3-phetio ()

molecules-and-light 1.5 ()

my-solar-system 1.3 ()

natural-selection 1.2 ()

natural-selection 1.3 ()

natural-selection 1.4 ()

natural-selection 1.5 ()

neuron 1.1 ()

normal-modes 1.0 ()

number-compare 1.0 ()

number-line-distance 1.1 ()

number-line-integers 1.2 ()

number-line-operations 1.1 ()

number-play 1.1 ()

ohms-law 1.4 ()

pendulum-lab 1.0 ()

ph-scale 1.6 ()

ph-scale-basics 1.5 ()

ph-scale-basics 1.6 ()

plinko-probability 1.1 ()

projectile-motion 1.0 ()

proportion-playground 1.0 ()

quadrilateral 1.1 ()

ratio-and-proportion 1.2 ()

reactants-products-and-leftovers 1.2 ()

resistance-in-a-wire 1.3-phetio ()

resistance-in-a-wire 1.6 ()

rutherford-scattering 1.1 ()

sound-waves 1.1 ()

states-of-matter 1.2 ()

states-of-matter-basics 1.2 ()

trig-tour 1.0 ()

under-pressure 1.1 ()

unit-rates 1.0 ()

vector-addition 1.0 ()

vector-addition-equations 1.0 ()

wave-interference 2.0 ()

wave-on-a-string 1.1 ()

waves-intro 1.1 ()

jonathanolson commented 6 months ago

Completed production releases, thanks! Closing.