suspicious that the code of the compression model saved in the published source code is wrong.

[alanMachineLeraning]

When inference, Top1 hits only 0.1, suspicious that the code of the compression model saved in the published source code is wrong.

[pierrestock]

Hi alanMachineLeraning,

Sorry to hear that. Please follow the Code of Conduct while posting an issue: could you please provide us the exact command and setup you ran to obtain that score?

[alanMachineLeraning]

Evaluate score is 65 when training. But only 0.1 when inference alone. Maybe the code of save model is not correct?

[alanMachineLeraning]

Inference code: parser = argparse.ArgumentParser(description='Inference for quantized networks') parser.add_argument('--model', default='resnet18', choices=['resnet18', 'resnet50', 'resnet50_semisup'], help='Model to use for inference') parser.add_argument('--state-dict-compressed', default='./state_dict_compressed.pth', type=str, help='Path to the compressed state dict of the model') parser.add_argument('--device', default='cuda', choices=['cpu', 'cuda'], help='For inference on CPU or on GPU') parser.add_argument('--data-path', default='/data/Imagenet/imagenet/ILSVRC2015/Data/CLS-LOC/', type=str, help='Path to ImageNet dataset') parser.add_argument('--batch-size', default=512, type=int, help='Batch size for fiuetuning steps') parser.add_argument('--n-workers', default=20, type=int, help='Number of workers for data loading')

[alanMachineLeraning]

the code to save model is the same with your code

[pierrestock]

Thanks for your answer! If I understand correctly:

• The model you quantize when using quantize.py reaches 65% top1?
• You evaluate the model you just quantized with inference.py and do not get a good accuracy?

As a sanity check, could you test inference.py with the provided compressed model by running:

python inference.py --model resnet18 --state-dict-compressed models/compressed/resnet18_small_blocks.pth --device cuda --data-path YOUR_IMAGENET_PATH

Also, what is the exact command line you run for training and the one for inference?

[alanMachineLeraning]

The model you quantize when using quantize.py reaches 65% top1? yes！ You evaluate the model you just quantized with inference.py and do not get a good accuracy? yes！ could you test inference.py with the provided compressed model by running: python inference.py --model resnet18 --state-dict-compressed models/compressed/resnet18_small_blocks.pth --device cuda --data-path YOUR_IMAGENET_PATH Result is correct ！ top1=65.1

[pierrestock]

Alright, I found the problem!

To reduce storage, for my pretrained models, I only store two arrays for one BN layer by writing

BN(x) = a*(x - m) / s + b = (a/s) * x + (b - m/s)

This is something not done by default in the file quantize.py but that is assumed in the file inference.py.

The workaround for you is to replace rows 84-87 of inference.py by:

for layer in bn_layers:
    state_dict_layer = to_device(state_dict_compressed[layer], device)
    attrgetter(layer)(model).load_state_dict(state_dict_layer)

Could you try it out?

Thanks for your patience and sorry for the inconvenience!

[xiaodao2049]

I download the code and simply run

python inference.py --model resnet18 --state-dict-compressed models/compressed/resnet18_small_blocks.pth --device cuda --data-path YOUR_IMAGENET_PATH

and I get the same result --top1 acc is 0.1. And when I replace rows 84-87 of inference.py by:

for layer in bn_layers:
    state_dict_layer = to_device(state_dict_compressed[layer], device)
    attrgetter(layer)(model).load_state_dict(state_dict_layer)

get an error : keyerror of no keys "running_mean" and "running_var". Can you give me some advice? @pierrestock @alanMachineLeraning . thanks you.