Feedback and next steps on relaxed mode

[ngzhian]

Given the support for relaxed mode in #105, let's address some feedback we got during Dan's presentation, and discuss next steps.

Some feedback from meeting notes

• changing NaN bits semantics in relaxed mode (Marat)
• rename relaxed_madd/relaxed_nmadd to alternate_fma/alternate_fnma (Marat)
• 'I think it will be helpful to be clearer on exactly what is meant by "mode," i.e. how it will be specified, documented by consumers, and used in practice.' (Thomas)

(Discussions for BF16 dot product should go to #106)

[Maratyszcza]

I like the idea of making NaN semantics deterministic, but feel it is out of scope of the Relaxed SIMD proposal.

[conrad-watt]

My (personal!) opinions:

• I prefer the current "relaxed" names to "alternate"
• @sunfishcode's "mode" idea should be realised through @rossberg's "profiles" proposal
• the "relaxed/non-deterministic" mode should be a clear default - I would personally be fine with relaxed SIMD phase 4 even before profiles are finalised (especially if we drop BF16 dot)
  • profiles could introduce the "strict/deterministic" profile, incorporating non-SIMD aspects like NaN bit determinism
  • anyone who produces relaxed SIMD before profiles is effectively saying they're targetting the "relaxed" (name bikeshedding: "default/full") profile

[eqrion]

* the "relaxed/non-deterministic" mode should be a clear default - I would personally be fine with relaxed SIMD phase 4 even before profiles are finalised (especially if we drop BF16 dot)

What does it mean for "relaxed" to be the clear default? My understanding from the presentation was that even if relaxed is a mode in the appendix, all toolchains and users are expected to conservatively assume that they may be run on a platform that uses relaxed mode.

I would guess the argument for making it a 'default' in spec language is to signal this is the desired long-term behavior, and to ensure that 'relaxed mode' has an ability to be removed easily in the future if hardware support becomes widespread enough. But this just all framing I suppose.

I also meant to say in the original presentation that we could see certain browsers (like Tor) adopt strict determinism (including NaNs). So I don't think this is just a 'cloud vendor' thing, if that changes the calculus.

[conrad-watt]

@eqrion part of my reasoning is that we will likely see toolchains with at least the option of producing "strict" mode specific code. It's always safe to produce code targetting the "relaxed" mode, which will then still work correctly in the "strict" mode, so we should be strongly encouraging at least the user-facing side of Wasm to clearly signal this as the default.

[dtig]

Circling back to this thread after the CG meeting to discuss next steps. The CG consensus was to not block on profiles, but there are a couple of things that might be worthwhile to discuss:

• How should the notion of a mode be surfaced in the Spec?
• If there is a deterministic mode, should it also have a deterministic FMA? (Perhaps a poll here would be useful?)

[ngzhian]

• How should the notion of a mode be surfaced in the Spec?

Currently we specify only the relaxed mode operations: https://www.ngzhian.com/relaxed-simd/core/exec/numerics.html#relaxed-operators

We would need to specify the deterministic variation of this. The actual semantics of the operators will be "subscripted" by the mode. I'm thinking that in the Numerics section, right now we have a "Relaxed operator" header. It can be divided into 2 subsections, "Relaxed operator in deterministic mode", and "Relaxed operator in relaxed-mode".

The summary of the section can be along the lines:

"Relaxed operators are operations whose definition depend on the host environment. The host environment can provide either a deterministic mode, or a relaxed mode. Relaxed operators are specified based on this mode."

Then proceed with definitions of deterministic relaxed operations, and relaxed-mode relaxed operations.

[penzn]

• If there is a deterministic mode, should it also have a deterministic FMA? (Perhaps a poll here would be useful?)

I think it was proposed deterministic FMA to be mul+add.

From acceptance criteria angle, would relaxed mode be required for browsers, i.e. is it OK if one Web implementation choses defaults to relaxed mode, while another - to strict?

[ngzhian]

• If there is a deterministic mode, should it also have a deterministic FMA? (Perhaps a poll here would be useful?)

I think it was proposed deterministic FMA to be mul+add.

Don't think so, see slide 4 of https://sunfishcode.github.io/RelaxedMathMode.pdf. FMA is deterministic, relaxed mode allows FMA or mul+add.

[penzn]

Don't think so, see slide 4 of https://sunfishcode.github.io/RelaxedMathMode.pdf. FMA is deterministic, relaxed mode allows FMA or mul+add.

Oh, that is surprising! I guess the motivation is that strict mode would run on newer hardware which has FMA...

[rossberg]

@ngzhian:

The actual semantics of the operators will be "subscripted" by the mode. I'm thinking that in the Numerics section, right now we have a "Relaxed operator" header. It can be divided into 2 subsections, "Relaxed operator in deterministic mode", and "Relaxed operator in relaxed-mode".

I think it is important that deterministic mode is not an alternative semantics, but a restriction of the general semantics, i.e., a subset of rules. So rather than 2 sections, there ought to be a single definition with some rules marked as unavailable in deterministic mode. Notably, all rules are applicable in non-deterministic mode.

Rationale: This makes sure that both semantics are consistent, and moreover, trivially ensures that all programs behaving correctly in non-deterministic mode also behave correctly in deterministic mode.

[ngzhian]

I think it is important that deterministic mode is not an alternative semantics, but a restriction of the general semantics, i.e., a subset of rules. So rather than 2 sections, there ought to be a single definition with some rules marked as unavailable in deterministic mode. Notably, all rules are applicable in non-deterministic mode.

Good point, here's my attempt:

   relaxed\_madd_N(z_1, z_2, z_3) &=& [ \fadd_N(\fmul_N(z_1, z_2), z_3), fma_N(z_1, z_2, z_3) ] & (\relaxedmode) \\
   relaxed\_madd_N(z_1, z_2, z_3) &=& fma_N(z_1, z_2, z_3) & (\otherwise) \\

where \relaxedmode expands to relaxed mode only (or something along those lines).

Somewhere in a paragraph above, we will have to describe the host as being in relaxed mode or not.

[conrad-watt]

Just to bikeshed a little, I'd prefer if the "relaxed" semantics is called something like "full"/"general"/"default" in the spec, and the deterministic semantics is called something like "strict"/"deterministic", especially if we aim to (for example) narrow the allowed NaN bit patterns of existing scalar float instructions in "deterministic" mode.

[rossberg]

@ngzhian, something like that, but without an "otherwise" and inverting the terminology like @conrad-watt suggested. I would also keep these annotations separate from side conditions, since they are a different dimension, and some rules could have both side conditions and annotations. I envision something like:

\notindetmode & relaxed\_madd_N(z_1, z_2, z_3) &=& [ \fadd_N(\fmul_N(z_1, z_2), z_3), fma_N(z_1, z_2, z_3) ] \\
  & relaxed\_madd_N(z_1, z_2, z_3) &=& fma_N(z_1, z_2, z_3) \\

Let me try to flesh out something for NaNs next week, so that you can perhaps follow that pattern.

[sunfishcode]

I chose the phrasing "relaxed-math mode" in my presentation because I proposed we use it to provide some level of containment around the list nondeterminism, rather than embracing it and having the deterministic mode be a subordinate subset of it.

However, if we end up structuring the spec in terms of profiles, then yes, it likely needs to be structured in terms of the common superset and subsets. I would still prefer to avoid the implied value judgement of calling it "default" though.

[ngzhian]

I rebased relaxed-simd changes on top of https://github.com/WebAssembly/profiles/pull/5, changes are at https://www.ngzhian.com/relaxed-simd/core/syntax/instructions.html#vector-instructions see https://github.com/WebAssembly/relaxed-simd/pull/115 for the code changes

Summary of changes:

• use profiles to exclude relaxed instructions in syntax/instruction
• this means that in Deterministic profile, the instructions don't exist (CompileError I think)
• no significant changes to validation
• no significant changes to execution, "Relaxed operators" contains the description of relaxed operations, no need to mark them as excluded in deterministic profile because "Any validation or execution rule that handles a marked construct is omitted by extension."
• text and binary format of instructions updated to include relaxed instructions

@rossberg @conrad-watt @sunfishcode fyi

[rossberg]

Hm, I thought the idea was that deterministic mode assigns deterministic semantics to these, rather than forbidding the instructions?

[ngzhian]

We discussed that during the sync, the concern is that not all instructions will have a satisfactory deterministic semantics, so it can be confusing if we have some that are map to deterministic semantics, and some that don't.

Now that I look at the list of instructions again, the only instruction that does not have a mapping to SIMD 128 instruction is FMA, since we don't have a FMA instruction.

(The other tricky one is #77 but we voted to not include that.)

I think we can switch it to assigning deterministic semantics.

[ngzhian]

Updated the semantics, the bracketed from only appears beside a !DET, this makes it clear that deterministic profile does not have any choice of projection.