Accuracy of sigmoid layer's output drops a lot

NVIDIA / Deep-Learning-Accelerator-SW

NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.

Other

181 stars 15 forks source link

Accuracy of sigmoid layer's output drops a lot #23

Open Railcalibur opened 6 months ago

Railcalibur commented 6 months ago

platform: Jetson AGX Orin 64GB OS: 5.1.2 DLA: 3.12.1

Sigmoid layers are used as the output of the model and the input & output shape of sigmoid is (8, 3, 88, 160). I found the accuracy of fp16 dla model drops a lot when I use sigmoid as output layer. However, the outputs is consistent to torch outputs if the sigmoid is removed, with the cosine similarity close to 1.

I want to know what is the limitations on the use of sigmoid layers ? Why does this loss of precision occur ?

nvoliver commented 5 months ago

@Railcalibur Is this issue still occurring with the latest JetPack 6.0? Also, does it occur with batch size 1, such that the shape is (1, 3, 88, 160). Thanks!