search

prs-eth / FILM-Ensemble

[NeurIPS 2022] FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation
MIT License
23 stars 2 forks source link

Using FiLM-Ensemble with different input #2

Closed JackFrost67 closed 1 year ago

JackFrost67 commented 1 year ago

First of all, this work is fantastic. Secondly, the paper has not mentioned any experiment using FiLM-Ensemble with different inputs (like point clouds, for example).
I'm currently working on uncertainty estimation in place recognition and I'm using explicit ensemble to do that. With FiLM, I can create an implicit ensemble (actually using PointNet as the backbone and aggregation layer like NetVLAD), but I have a problem with GPU memory overflow. To my understanding, the problem comes from during the forwarding pass, where there is the repeat_interleave call. Is this the efficient way to feed the model with CBN layers? Have you thought about more memory-efficient ways to feed the model?

0zgur0 commented 1 year ago

Hi, thank you for your interest in our work. It is straightforward to extend the algorithm to any data for instance as you described with PointNet for point clouds. But unfortunately, we haven't thought about more efficient implementation.