Reduce the tolerance used to checking correctness. Previous big difference is mainly caused by the format fp8e4m3b8, which introduced big rounding error in converting subnormal values. The change is to avoid subnormal values when generating inputs.
With that workaround, we can use much less tolerance in checking the correctness.
Reduce the tolerance used to checking correctness. Previous big difference is mainly caused by the format
fp8e4m3b8, which introduced big rounding error in converting subnormal values. The change is to avoid subnormal values when generating inputs. With that workaround, we can use much less tolerance in checking the correctness.