Open llCurious opened 2 years ago
I checked this blog . Currently, (32+29-11)/4 = 9.75 and (33+29-11)/4 = 10. Therefore, the input shall be resizes to 33*33?
I think for CIFAR-10 over ALexNet, the model architecture shall be modified? The first CNNLayer should use filter_size=7, strides=2, padding=2?
The model architecture for AlexNet is different from CryptGPU. For example, the second CNNLayer is 5596 as input, while in CryptGPU, it is 4496. Is there any mistakes here?
I haven't written an dataset loader for CIFAR-10 or ImageNet but I would imagine it is not too hard to write one (using either the C++ reference code for MNIST or using a few lines of PyTorch/TensorFlow).
You're right about the reason for the input size to be set to 33. That is because C++ requires sizes to match up precisely and when I was looking at references to AlexNet, that was the best way I could reconcile this gap.
There are a number of references for the head architecture of AlexNet when working with CIFAR-10. As far as I understand there is no right or wrong architecture. The different head architectures do make a bit of a difference from the ML side (change the overall training/inference accuracy) but for the focus of this project a certain choice was made.
CryptGPU is a more recent work and thus it is reasonable to say that their choice might be more thought out.
Yep, i have implemented the data loader for CIFAR-10 and ImageNet similar to MNIST.
And you are right that the model architecture does not influence the MPC side efficiency. I try to obtain reasonable accuracy for the private training of AlexNet over CIFAR10. Hence i have to figure out the reason for the mismatch of input shape and make sure the training over the given architecture can converge.
Currently, I have modified the code and model architecture according to the impl of torch, so that the AlexNet model load CIFAR-10 with 32x32x3 and ImageNet with 64x64x3. And there is no need to change input data shape. Thanks anyway!!
Hey @snwagh .
I try to test the AlexNet and VGG16 which takes CIFAR-10 and ImageNet datasets as training data. Is there data loader for these two datasets?
And i notice that the input size of CIFAR-10 for AlexNet is modified to 33x33, which is here. I wonder why this modification is made and how does this affect the dataloader, zero padding the raw input from 32x32 to 33x33?