WebXR Lighting Estimation

[alcooper91]

HIQaH! QaH! TAG!

I'm requesting a TAG review of the WebXR Lighting Estimation API

"Lighting Estimation" is implemented by AR platforms using a combination of sensors, cameras, algorithms, and machine learning. Lighting estimation provides input to rendering algorithms and shaders to ensure that the shading, shadows, and reflections of objects appear natural when presented in a diverse range of settings.

• Explainer: https://github.com/immersive-web/lighting-estimation/blob/main/lighting-estimation-explainer.md
• Specification URL: https://immersive-web.github.io/lighting-estimation/
• Tests: (will be a folder under https://github.com/web-platform-tests/wpt/tree/master/webxr)
• Security and Privacy self-review: https://github.com/immersive-web/lighting-estimation/blob/main/security-privacy-questionnaire.md
• GitHub repo (if you prefer feedback filed there): https://github.com/immersive-web/lighting-estimation/issues
• Primary contacts (and their relationship to the specification):
  • Brandon Jones, @toji, Google, Editor
  • Kearwood Gilbert @kearwood, (Former Mozilla), (Former?) Editor
  • Alex Cooper, @alcooper91, Google, Immersive Web WG Member
• Organization(s)/project(s) driving the specification: Immersive Web CG
• Key pieces of existing multi-stakeholder review or discussion of this specification: https://github.com/immersive-web/lighting-estimation/issues/
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): https://chromestatus.com/features/5704707957850112, General Mozilla support for WebXr specs (though Lighting not specifically called out): https://mozilla.github.io/standards-positions/#webxr

Further details:

• [X] I have reviewed the TAG's API Design Principles
• Relevant time constraints or deadlines:
• The group where the work on this specification is currently being done: W3C Immersive Web CG
• The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): W3C Immersive Web
• Major unresolved issues with or opposition to this specification: N/a
• This work is being funded by: N/a

You should also know that...

N/a

We'd prefer the TAG provide feedback as (please delete all but the desired option):

🐛 open issues in our GitHub repo for each point of feedback

[torgo]

Hi @alcooper91! Thanks for this review request. We note that the response to privacy & security section is great. One thing we felt was missing was the mitigation information on fingerprinting- which is actually discussed in the privacy & security considerations section of the explainer. And it also seems like there are requirements in the explainer - e.g. quantization - which do not actually appear in the spec itself? From a functional point of view the proposal seems like a great addition to XR.

[alcooper91]

Hi @torgo, I can certainly add a line to the Privacy section of the spec recommending quantization of the Spherical Harmonics and Primary Light Direction, I believe that was just a simple omission when trying to distill the key requirements from the explainer; but I'm not sure I understand what the first ask (about the fingerprinting mitigations) is, assuming it is separate from adding the quantization. Are you referring to the "How does this specification deal with sensitive information?" section of the security and privacy questionnaire?

Apologies for not responding sooner, it seemed that the only request was to simply add the additional quantization information.

[alcooper91]

immersive-web/lighting-estimation#45 Adds the requested line about quantization to the spec.

[torgo]

Thanks for that! To clarify, the quantization was the same as the fingerprinting mitigation issue, yes. Thanks for responding to this feedback and being aware of these issues.

[torgo]

We're happy to close this off now. Thanks for filing this and let us know if we can be of any more help.