search

facebookresearch / d2go

D2Go is a toolkit for efficient deep learning
Apache License 2.0
842 stars 202 forks source link

QAT, latency stays the same #230

Open maiiabocharova opened 2 years ago

maiiabocharova commented 2 years ago

Trained a model according to documentation. Added those configs lines for QAT

    cfg.QUANTIZATION.QAT.BATCH_SIZE_FACTOR = 1.0
    cfg.QUANTIZATION.BACKEND = "fbgemm"
    cfg.QUANTIZATION.QAT.FAKE_QUANT_METHOD = "default"
    cfg.QUANTIZATION.QAT.START_ITER = 1200
    cfg.QUANTIZATION.QAT.ENABLE_OBSERVER_ITER = 1200
    cfg.QUANTIZATION.QAT.ENABLE_LEARNABLE_OBSERVER_ITER = 1300
    cfg.QUANTIZATION.QAT.DISABLE_OBSERVER_ITER = 1200 + 300
    cfg.QUANTIZATION.QAT.FREEZE_BN_ITER = 12000 + 200

Trained for 1500 iterations.

The model is Ok and predictions are correct. But the model size and time of execution stayed exactly the same. Can you please help and advice on how to fix it?

wat3rBro commented 2 years ago

Are you running the exported torchscript moddel? the speed up and model size reduction is for running torchscript model on device.