How does locale get inside an XRSession?

[aphillips]

In discussing #1391 and thinking its implications, it occurs to me that there doesn't seem to be a way for locale to be communicated/set on an XR session (or other containers, such as views or such). How does the XR session find out what language/locale the user wants for localization (or other contextual information)?

I imagine this is actually a solved problem, but there is no mention of it in this spec. Shouldn't page authors have an interface for requesting the localized experience they want, e.g. to match the page hosting or linking to the XR experience?

(Because of timing issues, I'm going to file this as a personal comment now. The I18N WG will review it later and may promote it to needs-resolution at that time.)

[Manishearth]

The main question is what would be localized? XR sessions themselves do not really contain any localizeable content. They contain graphics, supplied by the app developer, who can choose to localize inside the rendered content.

The main localizeable content one may encounter within an XR session are permissions prompts, and "trusted UI" like the blurred menu when you temporarily exit XR, which are both typically controlled by the browser chrome. I don't think it is a Web norm for pagecontent to dictacte the localization of browser chrome, and I also don't think devices currently allow browsers that type of control.

(There's also a significant security issue with giving page content any control over the presentation of "trusted UI")

[aphillips]

How does the app developer know what localization to use for the rendered content? Is there no affordance for the request to include that information?

I get that the XR experience is "all just pixels", but some of those pixels might form shapes that are text or culturally appropriate images. As an app developer, how do I find out what locale to use to localize that content? How do I find out what locale the host site would like me to use?

I don't think it is a Web norm for pagecontent to dictacte the localization of browser chrome, and I also don't think devices currently allow browsers that type of control.

Page content and browser chrome integrate together in some places. There are active discussions about better integrating e.g. <input type=date> and browser calendar controls (to cherry pick an example) or as hints to match the localization of browser supplied native dialogs for APIs called from within a localized page experience (such as, I dunno, Payments?). The line is less distinct than it used to be.

I wouldn't say that this is control either, but a way of providing a hint: "hey, it would be nice if the session were localized in French, because this page is in French". Otherwise, is everything in navigator.language?

[Manishearth]

How does the app developer know what localization to use for the rendered content?

Same as how webapp developers know what localization to use for their website? Either explicit settings from the user, or Accept-Language/navigator.language.

This is not a new problem.

As an app developer, how do I find out what locale to use to localize that content? How do I find out what locale the host site would like me to use?

I don't understand, the "host site" and "app developer" are the same codebase. An XRSession is driven by Promises spawned by the host site, so the host site can just pass down any context it desires.

This question is roughly equivalent to the question of "how does an HTML <canvas> get told its locale", where the answer is also "the surrounding page which is the one actually driving the content, already knows the locale information it needs and can drive localization in the canvas[^1].

I think you might have a misunderstanding of how XR sessions work.

Page content and browser chrome integrate together in some places. There are active discussions about better integrating e.g. <input type=date> and browser calendar controls (to cherry pick an example) or as hints to match the localization of browser supplied native dialogs for APIs called from within a localized page experience (such as, I dunno, Payments?).

Yes, however these are discussions about specific controls. There are no such "specific" controls provided by the browser in XR sessions. There may be icons/etc for trusted UI, and then there's the trusted UI itself, but trusted UI is more equivalent to browser context menus than date pickers: you absolutely do not want to give content control over that, and the standard doesn't really dictate what goes in these menus to any level of specificity such that it could be localized.

Fundamentally, what is the thing being localized? Within an XR session, here's what can happen that's potentially localizeable:

• The user sees content that is made up of pixels provided by the application. The application already has control over this and can do what it likes.
• The user sees trusted UI, like the session pause menu or a permissions prompt. This is more like a context menu and there are security implications of providing the application control over the localization. This should use the browser chrome localization.
• The user sees OS trusted UI when pausing a session. This is the equivalent of hitting the start button or Ctrl-Alt-Delete; the browser has no control over this anyway.

What are you imagining as localizeable content here?

[^1]: There's a small caveat with this analogy: <canvas> supports higher level text APIs, where locale does matter. There's no such higher level API in WebXR

[aphillips]

I don't understand, the "host site" and "app developer" are the same codebase. An XRSession is driven by Promises spawned by the host site, so the host site can just pass down any context it desires.

Right. So every application developer is free to include a way for the locale to be passed in to an XRSession, but there is no affordance provided by XRSession to make this consisent.

How does the app developer know what localization to use for the rendered content?

Same as how webapp developers know what localization to use for their website? Either explicit settings from the user, or Accept-Language/navigator.language.

This is not a new problem.

I'm not claiming it is a new problem. I'm asking how it should happen.

Suppose I have a page in French with a button to launch an XR experience. How do I (as the invoker writing the Web page) tell my XRSession to use the French localization? Sure, I'm the developer of the session too, but do I really have to invent my own mechanism for that?

• The user sees content that is made up of pixels provided by the application. The application already has control over this and can do what it likes.

Agreed. I'm asking if XR should or already does provide a way for XR app developers to get this consistently vs. rolling their own. Accept-Language or navigator.language are not the same thing as the page language.

• The user sees trusted UI, like the session pause menu or a permissions prompt. This is more like a context menu and there are security implications of providing the application control over the localization. This should use the browser chrome localization.
• The user sees OS trusted UI when pausing a session. This is the equivalent of hitting the start button or Ctrl-Alt-Delete; the browser has no control over this anyway.

No argument with these.

I think you might have a misunderstanding of how XR sessions work.

Probably not, but I want to make allowances in case I am wrong. I have read one specification one time and so am aware I probably have blind spots.

[Manishearth]

but there is no affordance provided by XRSession to make this consisent.

Why must it be consistent? What benefit is derived from such consistency? Why is this one piece of information privileged in such a way that it needs a special conduit into the session, when there already is a generic conduit for any and all information the session needs to render its content?

Let's say I'm writing an XR game. I'll probably want to know the locale. I'll also probably want to know the user's name, and their avatar preferences, among many many other pieces of game information. All of this is going to be communicated at session startup somehow, probably passed in as a structured bag of game config/state. Why is locale special? Isn't it counterproductive to ask developers to pull that out of their game config?

How do I (as the invoker writing the Web page) tell my XRSession to use the French localization? Sure, I'm the developer of the session too, but do I really have to invent my own mechanism for that?

I don't understand why there is a "mechanism" here at all. It's just passing information around, in ways standard to Javascript.

I feel like this is equivalent to asking asking "how do I, as the invoker writing my web page, tell a function I am invoking to use the French localization"? There is no reason for there to be a consistent specced answer for that, and of course there currently isn't. There is no such specified mechanism for canvas, or requestAnimationFrame, or fullscreen, or any other API that behaves somewhat like XRSession. Why is XRSession special here?

Agreed. I'm asking if XR should or already does provide a way for XR app developers to get this consistently vs. rolling their own. Accept-Language or navigator.language are not the same thing as the page language.

I understand they are not the same. There is no current specced "consistent" way of talking about page language, developers always roll their own, and that's quite understandable since page language may need to be set locally, or on the server side, or using other signals. There may be some potential for an i18n standard that allows pages to talk about page language more consistently (window.locale or something?), but that is completely out of scope for this WG.

Probably not, but I want to make allowances in case I am wrong

In that case, I'd like to understand how in your eyes XRSession is different from WebGL content on <canvas> such that XRSession needs a specialized specced mechanism to pass down the locale whereas WebGL content does not. Or other requestAnimationFrame content, or fullscreen content, or web workers, or other cases where JS can invoke some more JS to produce a "different kind of content".

In fact, I'd go as far as saying that given that XRSession uses WebGL under the hood; if the thing you are asking for is indeed something that should exist (which I am still far from convinced on), it is actively bad architecture for WebXR to be the spec to design it in. Put differently: I think it is extremely weird for WebXR to be the first place this question is asked, and I think us working on this problem will lead to an extremely bad solution if it is in fact a real problem.

[aphillips]

A simple answer to my question appears to be: app developers are expected to define locale inputs for their application as needed.

Why must it be consistent? What benefit is derived from such consistency? Why is this one piece of information privileged in such a way that it needs a special conduit into the session, when there already is a generic conduit for any and all information the session needs to render its content?

The usual answer to this is: for most runtime environments, the locale is expected to be contextual. Wiring it up contextually saves every developer everywhere from having to think about it and ensure that the information is present. Unlike some values that can be read off the runtime (default timezone, to pick an example) or which are application specific (user name might be an example), the locale can be sourced from different contexts (navigator.language, calls to system locale APIs, or, for Web applications, defined from content inside the browser), providing a mechanism that the app developer can read for the locale (and which can be automatically wired up in XR frameworks, so they don't have to read it explicitly) might be useful.

If WebXR disagrees, just say so and resolve the issue. I asked the question not to prosecute a specific worldview but to honestly find out if you'd thought about the problem and had an answer. Also happy to meet with you folks at TPAC if there is interest in a high-bandwidth timeboxed conversation.

[Manishearth]

A simple answer to my question appears to be: app developers are expected to define locale inputs for their application as needed.

Yes, that's accurate.

Unlike some values that can be read off the runtime (default timezone, to pick an example) or which are application specific (user name might be an example), the locale can be sourced from different contexts (navigator.language, calls to system locale APIs, or, for Web applications, defined from content inside the browser), providing a mechanism that the app developer can read for the locale (and which can be automatically wired up in XR frameworks, so they don't have to read it explicitly) might be useful.

Given the framing, I still feel like I'm missing something here. Locale can be sourced from different contexts, but the current consensus of the Web Platform appears to be that that is intentional design to leave that choice up to the application developer.

As I said before, nothing on the Web is consistent about this now, how does consistency introduced here but nowhere else lead to anything useful?

which can be automatically wired up in XR frameworks

I remain confused as to what this means. What does this wire to on the other end, given that application locale may source from navigator.language, client-side settings, or server-side heuristics/settings, at the app developer's choice?

Perhaps you could sketch an example setup where an application does this?

If WebXR disagrees, just say so and resolve the issue. I asked the question not to prosecute a specific worldview but to honestly find out if you'd thought about the problem and had an answer. Also happy to meet with you folks at TPAC if there is interest in a high-bandwidth timeboxed conversation.

I'm not going to be at TPAC but happy to discuss this over a call. It's more that I'm finding the question here somewhat baffling because it is the first time I have seen this type of thing expected of a Web API, and I work in internationalization.

There's a mismatch of expectations here and I'd like to understand where it comes from. I've asked a bunch of questions in my previous comment above to try and tease that appart.

[Manishearth]

Closing the issue since the question was answered, however I'd still like to understand the mismatch. I don't want to have to wait until i18n WG review to hash this out.

[Manishearth]

Alternatively, I believe @toji will be at TPAC and can discuss this.

[toji]

I'm attending TPAC remotely this year, sorry.

[aphillips]

I don't want to have to wait until i18n WG review to hash this out.

This is part of your I18N review 😉

I'm not going to be at TPAC but happy to discuss this over a call. It's more that I'm finding the question here somewhat baffling because it is the first time I have seen this type of thing expected of a Web API, and I work in internationalization.

Let's schedule a call after TPAC then. I saw immersive on the calendar today, but was already booked (plus the power thing).