tc39 / proposal-measure

MIT License
12 stars 2 forks source link

Is the precision with respect to the scalar value, or the formatted value? #17

Open eemeli opened 3 days ago

eemeli commented 3 days ago

With mixed units, we're liable to end up representing e.g. 5ft 10in as something close to 5.83333, which when formatting needs precision to be defined, so that we don't end up with "5ft 9.99996in".

As with the scalar value that's discussed in #11, we need to determine whether the precision would be indexed by the smallest or largest unit:

  1. precision: 2 if by the largest unit
  2. precision: 0 if by the smallest unit

Even if we end up holding the value indexed by the largest unit, I'm starting to think that the precision ought to be measured as fractional digits of the smallest unit. Otherwise, we end up needing to come up with way too many solutions to corner-cases like what precision: 1 might mean in the above, or what precision: 2 means when formatting a value like "5ft 1in".

A key realisation here (at least for me) is that the total precision of a measurement should take into account the units: when formatting with foot-and-inch, there's a built-in expectation of the value including inches, which isn't present when formatting foot units. Therefore, the precision is most useful if it's considered to be going beyond the smallest declared unit.