Open PhilMiller opened 2 years ago
I think the ideal outcome of this would be to fold the scaling factors from right to left, with types as if the fold were initialized by element_type{1.0}
-- i.e. its multiplicative identity. For cases where element_type
is an arithmetic type, we could just do that.
I don't think the standard specifies a way to ask for the multiplicative identity of an arbitrary user-defined number type. As you've noted elsewhere, you'd like to avoid adding type traits for random numeric bits and bobs to the standard as part of specifying std::linalg
.
It would be workable but somewhat unprincipled to convert the right-most scaling factor to decltype(scaling_factor * declval<ElementType>())
, and then proceed from there - the conversion could hypothetically entail something weird, rather than preserving the value as best as possible.
Conversely, how cautious are users expected to be in not inadvertently widening their calculations?
If one had mdspan<float, ...> m
, and said scaled(2.0, m)
, rather than scaled(2.0f, m)
, that's going to come out as double
, and all of the subsequent arithmetic will run at half the throughput.
That seems like a really big, likely potential pitfall.
@crtrott
Note this only became a problem due to the attempt to undo multiple nested scalings, if we wouldn't try to undo it you are back to the () case. Furthermore, we can fix this by just defining the ProductScalingFactor differently: decltype(alpha * (x.scaling_factor() * declval(ElementType)))
done. This is not that hard ...
Regarding the widening: the design very explicitly added ScalingFactor as a way to change the type you do calculation in. Say you store your vector in bhalf or something like that, but now you want to scale it out, that requires widening, potentially.
If we really don't want that (and we can ask LEWG on this, I am not really set in stone on this part) we can just get rid of the ScalingFactor template parameter everywhere, and say the ScalingFactor we store and use to compute is ElementType.
If one had
mdspan<float, ...>
m, and saidscaled(2.0, m)
, ....
double
might even improve accuracy without reducing performance too much.scaled(2.0, A)
that 2.0 fits without rounding in float
(or that 2.0 is a commonly encountered constant) and dispatch to an all-float
BLAS optimized routine.double
, not just values that fit in float
. See [linalg.reqs.flpt] (though determining this would require a careful reading of "logarithmically stable").I would prefer not to override existing behavior of C++ arithmetic types. If the user gives us double * float
, we think they should get what the language gives them. I don't want to have to look into some custom traits hierarchy to figure out the value type of scaled(2.0, A)
for a matrix A
of float
.
My concern is about users getting upgraded/mixed precision inadvertently, because the parent interface potentially makes it too easy to do so.
For contrast, the classic DGEMM and SGEMM take their scalar coefficients in the same type as the underlying matrices, in both the Fortran and C APIs. Following established practice in that respect would suggest that the element type of a scaled accessor should match the element type of whatever it's scaling, and disregard the type of the scaling factor. In real arithmetic, I think that's likely to be fine, but if one wanted to scale a real matrix by a complex factor (why, I don't know), it wouldn't be.
Inspired by Numpy, would it make sense to offer an as_cast_to_type
accessor that could explicitly provide conversion for the intentional promotion case, as from real to complex, or int to float, and otherwise limit the type impact of scaling? EDIT: I think such an accessor should be produced by a function analagous to scaled
called converted<T>
The mixed-precision BLAS specifies all scalar arguments in double precision for the mixed-precision cases, and in precision matching the vectors/matrices in the mixed real/complex cases. In no case does the type of the scalar parameters drive the precision of the computation.
@mhoemmen regarding implementation freedom, I think the type impact of applying a scaled accessor is definitely something that should be specified and consistent across platforms, rather than something that implementations will decide arbitrarily. Otherwise, users who want a particular behavior will be unable to spell that intention in a portable fashion.
All of my input here is at least partially colored by recent work I've been doing on implementing mixed-precision configurations of an application, where some parts are single, and other parts are double, and conversions happen at relatively carefully chosen boundaries. One thing that was pointed out to me is that float/double conversion is actually not especially cheap on Nvidia GPUs, contrary to my assumptions.
Consider mdspan<double,…> m; scaled(1<<20, scaled(1 << 20, m))
- theProductScalingFactor
isint
, and its value will overflow, yielding UB, rather than scaling by2^40
as it would appear it should, and readily could if the calculation were indouble
.
This will be fixed in R9. scaled_scalar
will do the product immediately, and the resulting value type will be decltype((1 << 20) * decltype((1 << 20) * declval<double>()))
, that is, double
.
Conversation with @mhoemmen where I initially brought this up: