search

ARM-software / ComputeLibrary

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.
MIT License
2.76k stars 767 forks source link

FP16 tanh activation has high relative error around 0 #1002

Closed jondea closed 1 year ago

jondea commented 1 year ago

Output of 'strings libarm_compute.so | grep arm_compute_version': present on main branch

Platform: All aarch64 when using CPU backend

Operating System: Linux/any

Problem description: Relative error of FP16 tanh is very high around x=0 (have seen ~20%, but may be higher closer to zero). FP32 tanh uses a different approximation around 0 to fix this, but it is not present in FP16.

This came up as an issue when trying to integrate ACL with oneDNN https://github.com/oneapi-src/oneDNN/pull/1452. I will investigate further to see what a reasonable maximum relative error would be.

jondea commented 1 year ago

Patch to fix this is in review here: https://review.mlplatform.org/c/ml/ComputeLibrary/+/8480

morgolock commented 1 year ago

Thanks for the fix @jondea