Adding MeshCNN to PyTorch Geometric

[ranahanocka]

It would be so awesome if you could add MeshCNN to PyTorch Geometric. We have just released the code (PyTorch)

[rusty1s]

Thanks for letting me know. I see what I can do, although I would appreciate a pull request if you find the time.

[jlevy44]

I second this! Nice repo @ranahanocka !

If I had more time, I'd help out!

[Omri-L]

Hi, I'm trying to implement MeshCNN with PyTorch Geometric. Since I'm new with PyTorch Geometric and the basic stuff of graphs I would appreciate your help and comments. I would like to implement the first layer which is the mesh convolution: According to the MeshCNN paper:

One important note is that each edge holds a feature vector, not each node, therefore we can think of it as the edges are the nodes in a dual graph:

I've planned to implement it as the following: edge_index - a 2d Tensor with shape of 2xE, it holds all the edge connections to each other. In the example above: [[0, 0, 0, 0, 0, 1, 2, 3, 4, ...], [0, 1, 2, 3, 4, 0, 0, 0, 0, ...]] x is a tensor with all the features for all the connections. In the message function I've created the actual features (e, |a-c|, a+c, |b-d|, b+d) and used nn.Linear layer instead of convolution layer. Next, an aggregation function applied - "add". Last, the update function will return the aggregation output.

Do you think this is the correct way (using linear layer + aggregation "add" instead of convolution)?

Thanks

[rusty1s]

Yes, Linear layer and "sum" aggregation is the way to go to implement MeshCNN IMO. However, as far as I can see, edge indices needs to be sorted to guarantee correct computation of e_1, e_2, ...

[Omri-L]

Hi, I am planning to send a pull request in one of the next weeks for MeshCNN layers. In order to make it easier for me, I will appreciate if you could advise about the following modules:

1. MeshConv - implements the mesh convolution layer according to the paper. It uses MessagePassing pytroch geometric module. Should be in "pytorch_geometric\torch_geometric\nn\conv\mesh_conv.py" (?)
2. Mesh class - a data structure which reads the data from an obj file and creates relevant data for internal use (see below modules).
3. MeshPrepapre class - reads an obj file, prepares the data for Mesh class such as extracting mesh features and creates mesh augmentations.
4. MeshUnion class - a utility class which unifies mesh edges and features.
5. MeshPool - implements the mesh pooling layer according to the paper. It uses a utility class named MeshUnion and a the structure class named Mesh. Should be in "pytorch_geometric\torch_geometric\nn\pool\mesh_pool.py" (?)
6. MeshUnpool - implements the mesh unpooling layer according to the paper. It uses Mesh class. Should be in "pytorch_geometric\torch_geometric\nn\unpool\mesh_pool.py" (?)

Mesh augmentations, Mesh features and Mesh utils should be on different files but are not classes.

My questions are:

1. Where should I put each class or file?
2. For which class should I add a unit test?
3. Should I create a full example of use of loading data and building a small network which uses the above modules? (such as in MeshCNN github).

Thanks!!!

[rusty1s]

Thanks for your interest in contributing.

1. Do we really need an own Mesh class or is Data sufficient?
2. I guess MeshCNNPrepare should be implemented as a transform in torch_geometric.transforms.

It would be great to have a unit test for MeshConv, MeshPrepare and MeshPool, but those are not mandatory. We can create them afterwards. A full example to show-case its usage would be awesome :)

[Omri-L]

1. Mesh class - I did not described the class functionality as it is. It is not just a data structure but it also have some functionality (such as adding or removing edges, vertexes, exporting data to file and more). In addition, it holds more fields (but I saw that Data can be extended).
2. I will look into it.

Thanks

[sharmishtaa]

I support this idea! I would love to see MeshCNN here. Are there any updates on when we might be merging?

[gattia]

@Omri-L

I notice you have a fork w/ some progress on mesh cnn. How close is that to functioning?

[daniel-unyi-42]

@Omri-L @rusty1s any updates on this? I think MeshCNN would be an important addition to PyG and offer my help to finish things up.