Unexpected Results On MiniImageNet With Pretrain ResNet

[KZF-kzf]

hello,Hi, I'm having some problems replicating your results. 2024-07-17 12:23:23,497 - meta_test.py[line:154] - INFO: max |k: 0.26 |mix acc: 74.56+0.65% |gap: 11.82 2024-07-17 12:23:23,497 - meta_test.py[line:156] - INFO: ACC:|proto acc: 62.74+0.80% |gen acc: 70.96+0.69% My experiment is set up as follows: 1.I used the ResNet-MinilmageNet.pth file provided in your cloud disk as the weight of compute_center.py 2.Execute train_seman_l1_cen.py to get the epoch_best-pth file, and then execute the meta-test, with the result shown above. 3.The other Settings remain the default or the same as you github gave Now there seems to be some gap between the results I got and those in the paper. Is there anything wrong with my Settings?

[KZF-kzf]

train.txt this is my train log

[DruryXu]

Sorry for the late reply. Could you please check whether the dataset used is correct?

[KZF-kzf]

Thank you very much for your reply. Here is the data set I used, and I think that should be fine. Please let me know if you need more details. Thank you, and for your excellent work.

[myl666666]

我看到作者的GUTHUB上的work-dir 是 resnet,你的好像是CNN

[DruryXu]

Thank you very much for your reply. Here is the data set I used, and I think that should be fine. Please let me know if you need more details. Thank you, and for your excellent work.

The way you use MiniImageNet in code is correct, but there are many different versions of the miniimagenet dataset, please confirm which version of miniimagenet you are using. If possible, please use the miniimagenet which is approximately 3GB in size.

[DruryXu]

我看到作者的GUTHUB上的work-dir 是 resnet,你的好像是CNN

They are the same thing with different names, we modified this name for the sake of consistency.

[KZF-kzf]

Thank you very much for your reply. Here is the data set I used, and I think that should be fine. Please let me know if you need more details. Thank you, and for your excellent work.

The way you use MiniImageNet in code is correct, but there are many different versions of the miniimagenet dataset, please confirm which version of miniimagenet you are using. If possible, please use the miniimagenet which is approximately 3GB in size.

Thanks for your reply.Can you give a download link of the MiniImageNet used in your code?

[MeycL]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

[DruryXu]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

Could you please share your reproduction results？

[MeycL]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

Could you please share your reproduction results？

Thank you very much for the prompt response.

In the miniimagenet experiments, the results for 5-way 1-shot are 71.87+0.68, while for another 5-way 1-shot it is 77.55+0.56. This clearly differs significantly from the results presented in the paper, and similar issues are observed in other datasets.

My experimental environment is: Ubuntu 20.04, PyTorch 1.11.0, CUDA: 11.3. All code and pre-trained weights are the latest versions, and the experimental steps follow the readme. Since this project does not provide the dataset used, I referred to FewTURE for my dataset. During my attempts, I noticed that the dataset seems to affect the results. While checking other issues, I found many questions related to the dataset. Could the authors provide a Google Drive link or another clearly accessible link to all the data (miniimagenet, tieredimagenet, cifar-fs, fc100) used? I believe this would help everyone's learning and the dissemination of this work.

Look forward to your reply.

[DruryXu]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

Could you please share your reproduction results？

Thank you very much for the prompt response.

In the miniimagenet experiments, the results for 5-way 1-shot are 71.87+0.68, while for another 5-way 1-shot it is 77.55+0.56. This clearly differs significantly from the results presented in the paper, and similar issues are observed in other datasets.

My experimental environment is: Ubuntu 20.04, PyTorch 1.11.0, CUDA: 11.3. All code and pre-trained weights are the latest versions, and the experimental steps follow the readme. Since this project does not provide the dataset used, I referred to FewTURE for my dataset. During my attempts, I noticed that the dataset seems to affect the results. While checking other issues, I found many questions related to the dataset. Could the authors provide a Google Drive link or another clearly accessible link to all the data (miniimagenet, tieredimagenet, cifar-fs, fc100) used? I believe this would help everyone's learning and the dissemination of this work.

Look forward to your reply.

This is a weird result, is this result comes from resnet or swin tiny? Thank you for reminding us, we will check whether the problem comes from bugs during our code reorganization. We will also consider the dataset suggestion, thank you.

[MeycL]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

Could you please share your reproduction results？

Thank you very much for the prompt response. In the miniimagenet experiments, the results for 5-way 1-shot are 71.87+0.68, while for another 5-way 1-shot it is 77.55+0.56. This clearly differs significantly from the results presented in the paper, and similar issues are observed in other datasets. My experimental environment is: Ubuntu 20.04, PyTorch 1.11.0, CUDA: 11.3. All code and pre-trained weights are the latest versions, and the experimental steps follow the readme. Since this project does not provide the dataset used, I referred to FewTURE for my dataset. During my attempts, I noticed that the dataset seems to affect the results. While checking other issues, I found many questions related to the dataset. Could the authors provide a Google Drive link or another clearly accessible link to all the data (miniimagenet, tieredimagenet, cifar-fs, fc100) used? I believe this would help everyone's learning and the dissemination of this work. Look forward to your reply.

This is a weird result, is this result comes from resnet or swin tiny? Thank you for reminding us, we will check whether the problem comes from bugs during our code reorganization. We will also consider the dataset suggestion, thank you.

Swin tiny.

[DruryXu]

Have you found all the correct datasets? (miniimagenet, tieredimagenet, cifar-fs, fc100) I have looked for publicly available data online to train, but I can't reproduce the results in the paper (all are lower). If possible, could you please provide the links to the correct datasets? Thank you very much.

Could you please share your reproduction results？

Thank you very much for the prompt response. In the miniimagenet experiments, the results for 5-way 1-shot are 71.87+0.68, while for another 5-way 1-shot it is 77.55+0.56. This clearly differs significantly from the results presented in the paper, and similar issues are observed in other datasets. My experimental environment is: Ubuntu 20.04, PyTorch 1.11.0, CUDA: 11.3. All code and pre-trained weights are the latest versions, and the experimental steps follow the readme. Since this project does not provide the dataset used, I referred to FewTURE for my dataset. During my attempts, I noticed that the dataset seems to affect the results. While checking other issues, I found many questions related to the dataset. Could the authors provide a Google Drive link or another clearly accessible link to all the data (miniimagenet, tieredimagenet, cifar-fs, fc100) used? I believe this would help everyone's learning and the dissemination of this work. Look forward to your reply.

This is a weird result, is this result comes from resnet or swin tiny? Thank you for reminding us, we will check whether the problem comes from bugs during our code reorganization. We will also consider the dataset suggestion, thank you.

Swin tiny.

We have uploaded datasets

[DruryXu]

Thank you very much for your reply. Here is the data set I used, and I think that should be fine. Please let me know if you need more details. Thank you, and for your excellent work.

The way you use MiniImageNet in code is correct, but there are many different versions of the miniimagenet dataset, please confirm which version of miniimagenet you are using. If possible, please use the miniimagenet which is approximately 3GB in size.

Thanks for your reply.Can you give a download link of the MiniImageNet used in your code?

We have uploaded datasets