Probabilistic Max Pooling

[nlahaye]

Hi Gustavo!

I hope you are doing well!

I was wondering if you had looked into / planned to add probabilistic max-pooling, as descibed in: H. Lee, et al., Convolutional Deep Belief Networksfor Scalable Unsupervised Learning of Hierarchical Representations Proceedings of the 26th annual international conference on machine learning (2009).

I am happy to help, but figured it would be best to consult you first.

Thanks! Nick

[nlahaye]

@gugarosa Just wanted to check in on this. Hope all is well!

Nick

[MateusRoder]

Hello @nlahaye. I hope is everything well with you, and thanks for your collaboration =D First of all, I think Gustavo is a little busy with his work, so he ended up not answering.

Regarding the implementation of the probabilistic max-pooling, as part of the development team, my answer is yes and no at the same time. Particularly, when I was working on papers and implementation about the conv-rbms, I was really excited to implement the method. However, I saw in most of the papers that the probabilistic max-pooling did not make improvements on the performance that could justify the implementation. At that time, my conclusion was that by restringing the hidden activation map to the binary approach taken by the prob-max-pool, we can lose important pieces of images information, mainly when dealing in a difficult domain and some conv-rbm on the stack.

Hooowever, academically speaking, would be amazing to have the method on learnergy, since we can explore its behavior on several real-world applications. At this point, with you want to gift the lib with your additional help on such implementation, would be a pleasure for us as community, feel free to do it. Also, feel free to contact us always, as usual.

Best regards, Mateus Roder.

[gugarosa]

Hi @nlahaye! I hope everything is going well with you.

As Mateus said, we even had this goal as a long-term in our roadmap: https://github.com/gugarosa/learnergy/projects/1, but we have not been able to do.

Please feel free to contribute if you have some time, that would be an amazing addition to the library!

Best regards, Gustavo.

[nlahaye]

Hi @gugarosa and @MateusRoder! Thanks for the info! I will keep that in mind ! I ask because I am working on implementing an RBM-based architecture similar to a Fully Convolutional Network, but for unsupervised feature extraction and segmentation. I am going to test it out without MaxPooling first, given your guys' previous results, and then I will work on implementing MaxPooling so we can have it in the repo and I can compare the differences in performance on my end.

Also (unrelated to this issue) I recently got the chance to work with a couple of folks from NVIDIA who provided some insight for optimizations and improvements to my research code and Learnergy. Once I clean up the changes and make sure they work as expected with and without usage of DDP, I hope to see if we can integrate them into the main repo.

Hope you both are doing well!

Nick

[MateusRoder]

Hello @nlahaye. It is a pleasure to follow your advances in the research, especially with the RBMs, congratulations!! =D I just realized that in older codes on my note (I was doing a deep clean) I wrote the maxpooling for the convolutional models, but I did not make the commit to the repo at that time. Since the code and the learnergy version is old, I am going to make intensive tests to verify the coherence. Once It is ok, I update this topic in our discussion.

Best Regards, Mateus.

[MateusRoder]

Nick, @nlahaye. Updating our topic... The MaxPooling2D was released for all convolutional models! =D

Best regards, Mateus.

[nlahaye]

Awesome, thank you so much for adding this in! Much appreciated!