search

janericlenssen / noise-as-targets-tensorflow

Noise-as-targets representation learning for cifar10. Implementation based on the paper "Unsupervised Learning by Predicting Noise" by Bojanowski and Joulin.
22 stars 7 forks source link

Improving MLP Test accuracy #1

Open abhijithchunduru opened 7 years ago

abhijithchunduru commented 7 years ago

Hi, I implemented this repository. As such, I get a mlp test accuracy of 42%. By directly connecting the mlp to the convolutional features instead of the representation layer, I could push the mlp test accuracy to about 52 %.The paper claims better test accuracy on tougher datasets like ImageNet and Pascal VOC. Please share some tips on improving test accuracy and get the results claimed in the paper. My plan is to eventually to eventually use this as a pre-training for image segmentation

Regards Abhijith

abhijithchunduru commented 7 years ago

I suspect an issue with the encoder and the target reassignment because my training loss for the encoder hardly drops under 1

janericlenssen commented 7 years ago

Thanks for your comments! Using the convolutional features instead of the learned representation is a good hint, although it is not what the authors claim they do. Unfortunately, I do not have any further tips for improving the results. I am not one of the authors of the paper and I am not in contact with them. I tried different variations that did not led to improvements and after a while I decided to use adversarial autoencoders instead because they seem to work better for the same task (at least for me). I plan to upload the tensorflow AAE implementation in the near future.

Regarding the issue: Can you clarify what you mean exactly? Judging by the loss function, the reassignment seems to work well. The loss is dropping fast during the first reassignments.

Regards, Jan