ranahanocka / MeshCNN

Convolutional Neural Network for 3D meshes in PyTorch
MIT License
1.58k stars 315 forks source link

Changing Mesh Pooling functions #8

Closed wofsauge closed 5 years ago

wofsauge commented 5 years ago

Hi, I want to change how MeshCNN collapses the edges in order to train it to reduce a mesh based on its normals. The smaller the difference between the input Triangle Normal and the Output, the more likely it is that the edge gets collapsed. The overall goal of me is to write a neural network that reduces the polygon count of any input mesh based on a set of criteria.

Where exactly can i find the code that decides whether or not a neuron should collapse an edge or not? i would guess that its in models/layers/mesh_pool.py and there somewhere in the "__build_queue" function. But since the code is not very well documented, i dont quite understand what its doing there. Any idea where i should try to implement that feature?

ranahanocka commented 5 years ago

Hi, I am not sure I understood the collapse criteria, is it determined completely outside of the network? In other words, is this collapse order determined regardless of the learned features ?

Anyway, yes you are correct, you should modify the code in models/layers/mesh_pool.py and specefically, you should sort the edges based on the criteria you determined in the __build_queue method. Right now we use a heap implementation for speed since it is dynamic, but if your collapse is static (i.e., it is determined outside of the network) the heap is not necessary. Right now , the heap returns a list of edge_ids which were sorted by the minimum edge magnitude.

To do minimum changes to the code and just get it running as you like (not necessarily efficiently), you should build a list of edge_ids in the priority you wish to collapse them.

For a more efficient implementation, you can add a property to the mesh class which has an edge priority list (for example, in mesh_prepare.py you can add an attribute called edge_priority, which the mesh_pool will use to pool edges)

Goodluck & let me know how it goes :)

ranahanocka commented 5 years ago

@wofsauge I'm marking this as closed for now, please reopen if some more clarification / followup is needed.

zideliu commented 3 years ago

@wofsauge I have the same idea as you, so I want to ask you if this problem has been solved? Thank you~

wofsauge commented 3 years ago

@zideliu hey. My problem could sadly not be resolved, because it was in general already too complex for a CNN of that time. I had little to no experience with neural network coding as well, so the results I was able to create where very limited. Since the whole project was for my thesis, I had a limited time frame, therefore after messing around for a few days, I sadly scrapped the idea, and focused on a non-AI based approach to solve the problem as best as possible.

zideliu commented 3 years ago

@wofsauge oh,Like you, I am also going to complete an undergraduate thesis on 3D-Mesh segmentation. To be shy, could I read your paper about this?

wofsauge commented 3 years ago

@zideliu my thesis is written exclusively in German and has a soft NDA on it since its written for a company. If you can read that language I can send you a copy in private, but I am not allowed to publish it on the internet.

zideliu commented 3 years ago

@wofsauge oh!!I really hope to be able to observe your thesis. Thank you very much, my email is lzdliuzide@gmail.com you help me a lot