search

Lyken17 / SparseNet

[ECCV 2018] Sparsely Aggreagated Convolutional Networks https://arxiv.org/abs/1801.05895
MIT License
125 stars 25 forks source link

Question on parameter count of sparsenet_d_40_k_12 on cifar10 #5

Closed Sirius083 closed 5 years ago

Sirius083 commented 5 years ago

In the paper, Table 2, the SparseNet_d_40_k_12 (no bottleneck) the total parameters of the model is 0.8M from the chart However I run your code (tensorflow) the log shows the total paramter count is 0.18M (I trained the model and generate the pb file to re-compute the total parameter, is still 0.18M) Do you know why there is inconsistency? Thanks in advance

Lyken17 commented 5 years ago

That's werid. What is your tensorflow version? I need to check whether it is caused by wrong implemention or TF API change.

Sirius083 commented 5 years ago

Thanks for you reply tensorflow:1.12 tensorpack:0.9.1 python: anaconda 3.6

One more question, did you train your model on cifar10 under tensorflow or pytorch? I trained the model of sparsenet_d_40_k_12(no bottleneck) on cifar10 the best validation error on test set is 7.19%

part of the logging

block3/densen_layer.9/bn1/beta:0    [48]               48
block3/densen_layer.9/conv1/W:0     [3, 3, 48, 12]   5184
block3/densen_layer.10/bn1/gamma:0  [48]               48
block3/densen_layer.10/bn1/beta:0   [48]               48
block3/densen_layer.10/conv1/W:0    [3, 3, 48, 12]   5184
block3/densen_layer.11/bn1/gamma:0  [48]               48
block3/densen_layer.11/bn1/beta:0   [48]               48
block3/densen_layer.11/conv1/W:0    [3, 3, 48, 12]   5184
bn_last/gamma:0                     [48]               48
bn_last/beta:0                      [48]               48
linear/W:0                          [48, 10]          480
linear/b:0                          [10]               10
Total #vars=119, #params=185778, size=0.71MB

the log in pytorch implemented model denselink.py is same as that in tensorflow (change depth=40, nClass=19, bottleneck=False, fetch="exp")

  (classifier): Sequential(
    (avgpool): AvgPool2d(kernel_size=8, stride=8, padding=0)
    (flattern): Flattern(dim=0)
    (linear): Linear(in_features=48, out_features=10, bias=True)
  )
)
  + Number of params: 0.18
torch.Size([12, 10]) tensor(-20.4898, grad_fn=<SumBackward0>)
Lyken17 commented 5 years ago

Thanks for pointing out. Actually the entry should be sparsenet-40-24 (as listed in README). I will fix the wrong text and upload a new arxiv version.