mozilla / federated-learning-v2-study-addon

Mozilla Public License 2.0
1 stars 2 forks source link

The "num_key_down_events_at_selecteds_first_entry" attribute value only changes to correct value after visiting 3 times the same complete URL #14

Closed SoftVision-PaulOiegas closed 5 years ago

SoftVision-PaulOiegas commented 5 years ago

[Affected versions]:

[Affected Platforms]:

[Prerequisites]

[Steps to reproduce]:

  1. Open Firefox browser with the profile from prerequisites and navigate to "about:telemetry" page.
  2. Open a new tab, enter the full URL for a website that was not visited yet (www.nike.com) and hit Enter key.
  3. Observe the telemetry attribute for "num_key_down_events_at_selecteds_first_entry" for the entered website.
  4. Close the tab with the website, open a new tab and navigate to the same full website URL.
  5. Observe again the telemetry attribute for "num_key_down_events_at_selecteds_first_entry".
  6. Close the tab, open a new tab and enter again the same full URL.
  7. Observe again the telemetry attribute for "num_key_down_events_at_selecteds_first_entry".

[Expected result]:

[Actual result]:

[Notes]:

motin commented 5 years ago

This took me a couple of retries to reproduce, but I see now that it is expected behavior. If you watch closely at 1:26 in the screen recording, you see that the topmost suggestion at the time is "http://www.adida - Visit" and the suggestion that eventually gets selected is "http://www.adidas.com - Visit" which first appears after 14 keydown events has been observed. At 1:46 (the third attempt), the first suggestion is "www.adidas.com - Visit" and it shows up at an earlier amount of keystrokes. My guess is that it shows up earlier precisely because at the third attempt, there are enough history visits for it to rank higher. Thanks for this detailed issue! I am going to close it since there is nothing wrong with the instrumentation in the study add-on, but it is a nice issue showing how the frecency algorithm affects the suggestions after repeated visits. Also, the highlighted awesome bar behavior may help the study analysts understand some of the metrics. :)