search

zhutmost / lsq-net

Unofficial implementation of LSQ-Net, a neural network quantization framework
MIT License
274 stars 40 forks source link

Issue with training Resnet20 on Cifar10 #28

Open Shanmukh-C opened 5 months ago

Shanmukh-C commented 5 months ago

I tried training resnet20 on cifar10 dataset and have some surprising results could you please let me know what might be going wrong (I have just changed arch and num_classes in config. This is my logger which shows accuracy 91% pre training and 99% just after an epoch which varies very drastically from your logger though it is trained on imagenet: image

zhutmost commented 5 months ago

Sorry that I dont have data and I didnt used these code (or even quantization) for a long time. I have refactored this repo as NeuralZip (https://github.com/zhutmost/neuralzip), where I posted some results of ResNet-50/18 over imagenet with 2/3/4bit.

BodongDu commented 5 months ago

(flashatt) yangyk@yyk-s1:~/yangyk/NN_CUDA/lsq/lsq-net$ python main.py ./examples/lsq/resnet20_a2w2_cifar10.yaml /home/yangyk/yangyk/NN_CUDA/lsq/lsq-net <class 'pathlib.PosixPath'> INFO - Log file for this run: /home/yangyk/yangyk/NN_CUDA/lsq/lsq-net/out/resnet20_a2w2_cifar10_20240531-171941/resnet20_a2w2_cifar10_20240531-171941.log INFO - TensorBoard data directory: /home/yangyk/yangyk/NN_CUDA/lsq/lsq-net/out/resnet20_a2w2_cifar10_20240531-171941/tb_runs Files already downloaded and verified Files already downloaded and verified /home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/utils/data/dataloader.py:560: UserWarning: This DataLoader will create 32 worker processes in total. Our suggested max number of worker in current system is 16, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary. warnings.warn(_create_warning_msg( INFO - Dataset cifar10 size: Training Set = 50000 (196) Validation Set = 10000 (40) Test Set = 10000 (40) INFO - Created resnet20 model for cifar10 dataset Use pre-trained model = True tensor(8) Traceback (most recent call last): File "main.py", line 120, in main() File "main.py", line 59, in main tbmonitor.writer.add_graph(model, input_to_model=train_loader.dataset[0][0].unsqueeze(0)) File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/utils/tensorboard/writer.py", line 841, in add_graph graph(model, input_to_model, verbose, use_strict_trace) File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/utils/tensorboard/_pytorch_graph.py", line 337, in graph trace = torch.jit.trace(model, args, strict=use_strict_trace) File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/jit/_trace.py", line 794, in trace return trace_module( File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/jit/_trace.py", line 1056, in trace_module module._c._create_method_from_trace( File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/home/yangyk/anaconda3/envs/flashatt/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1488, in _slow_forward result = self.forward(input, **kwargs) File "/home/yangyk/yangyk/NN_CUDA/lsq/lsq-net/model/resnet_cifar.py", line 125, in forward out = F.avg_pool2d(out, kernel_size=out.size()[3]) TypeError: avg_pool2d(): argument 'kernel_size' must be tuple of ints, not Tensor

How to fix this bug when I try to train resnet20 for cifar-10