search

Unity-Technologies / Unity.Mathematics

The C# math library used in Unity providing vector types and math functions with a shader like syntax
Other
1.38k stars 156 forks source link

Why does `math.sign` return the input type? #208

Open HellGate94 opened 2 years ago

HellGate94 commented 2 years ago

I noticed this difference when working on my own (mostly) compatible library. This is different than the System.Math and hlsl definition where they always return an integer type.

Any particular reason why this change is done?

unpacklo commented 2 years ago

This may have been a mistake if the return type is different from the HLSL version. Usually, I prefer to return the same type as the input for functions like this if there's a reasonable chance that someone would use the returned value as a part of the computation. So for instance, if you wanted to do x * sign(x) or something similar.

Are you running into a problem due to the fact that the return type is different from the other versions?

HellGate94 commented 2 years ago

I guessed the reason has to do with preventing type conversion, but was surprised that it is worth changing the default definition.

Is it currently causing problems? Not really.

Could it cause problems? When doing math and floating point comparisons maybe? Could cause implicit type conversions int * sign(float) => float

The hlsl definition:

Name Template Type Component Type Size
x scalar, vector, or matrix float, int any
ret same as input x int same dimension(s) as input x