Clarifying my understanding of the Shared Storage API proposal

[rwiens]

Hi! I'm trying to understand how this proposal might work for cross-domain A/B experiments (e.g.trying to enable the same treatment on the same subset of users across 2 different websites). I've described below what my current understanding is of how the Shared Storage API proposal works as well as some questions on parts I'm unsure of. Could you please help clarify any parts I misunderstood?

Shared Storage

Anyone can write to the shared storage but there are limits on who and what content can be read from it.

• Question: How do you prevent domains stomping on one another's content? And what prevents a company from reading a different company's content?

Worklets

Websites can also write functions (called "worklets") that Chrome will execute in-browser based on the contents of the shared storage. The worklets can edit the shared storage, trigger the aggregated reporting workflow, or return content in an opaque URL, but not anything beyond that.

• E.g. a party could write a unique identifier (called a seed) to the shared storage, and perform different treatment based on a modulus of the identifier. Put otherwise, the user could be assigned to one of N different treatment groups but would be consistently assigned the same treatment.

  • It seems like there might be very low limits on N, but it's unclear how N is determined. Further comments show N is expected to stay under 10.

  • Question: Why does N need to be so small? Is there room to scale larger if your base population is large enough? While a small number of bits can be identifying if you have a small number of users, it's possible to have large numbers that aren't identifying if there's a large number of users, as long as the numbers are evenly distributed. Given that this proposal already requires aggregated reporting, would it be feasible for someone to run more types of treatment and just wait longer until enough users are in each treatment group to safely generate the aggregate report?

• Is there any way to support multiple independent treatments with different user splits? E.g. if one treatment coloured the website background red and another treatment changed the font, could the users be split into these experiments independently such that some arbitrary subset of users might be in both?

• Anything the function calls or loads needs to be done via opaque URL. In the long term, everything loadable via opaque URL should be a blob that's downloaded ahead of time and available offline. Put otherwise, visiting this URL wouldn't result in a server call when the user's browser actually accesses it.

  • This likely means that the content within this URL should be limited to something that could be executed in browser. Which also means that it can't pass relevant info to the serving call to trigger server-side treatment changes. Thus it doesn't enable running user-consistent A/B experiments that affect server-side behaviour.

• "Operations defined by one context are not invokable by any other contexts." -> I'm not sure what this means. Does it mean we couldn't rerun the same treatment on different domains?

• Unclear how flexible these functions/worklets are aside from letting you pick between 1 of 5 opaque URLs. Can you actually change behaviour in these functions, or does the behaviour change need to be within the URL? If the former, can you apply different worklets to different situations, so that you get behaviour changes for some situations and URLs for others?

Aggregated Reporting

The only way to get info back about your treatment is sending metrics in aggregate.

• Seems like you can write as many metrics as you want as long as it's in key-value pairs, but you can only get aggregate results returned to your servers at some time interval.

• Is there any built-in support for splitting the metrics by treatment groups, or do you have to manually write each group to a different metric key?

• Will the metrics include confidence intervals?

[jkarlin]

Thanks for the questions!

Question: How do you prevent domains stomping on one another's content? And what prevents a company from reading a different company's content?

Data is stored per origin. So if you're in