libffcv / ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)
https://ffcv.io
Apache License 2.0
2.79k stars 180 forks source link

Compression error causes performance drop #364

Open erow opened 5 months ago

erow commented 5 months ago

Hi, I found a problem of FFCV, which may not the fault of the library, but this library suffers from it.

I use the same model to evaluate the accuracy on different validation sets. The results show compression reduces the performance significantly, but it's too much for FFCV. image

I applied FFCV to accelerate SSL. The performance dropped for both pretraining and finetuning stages.

We set the JPEG compression quality at 90.0% to maintain high visual fidelity, write mode to jpg, the maximal resolution to 1000.

First, I hope that a lossless compression is supported. Second, I wonder any solution to reduce the compression shidt.