Any ideas for better input features than HKS with code available?
How many epochs did you train the pre-trained models for to get such good performance?
Any ideas on modifying the architecture (e.g. the Point-wise MLP in the end to Attention block) and/or hyperparameters to get better performance?
Long version:
First of all, thanks so much for your (&CMU & Ecole Polotechnique labs) amazing work on DiffusionNet and for open sourcing the code along with pre-trained models!
I'm doing a course project with a friend about improving performance on the SotA (DiffusionNet in this case) FAUST correspondence somehow (bummer: the project deadline is soon), and we had some ideas on integrating other features and/or modifying the architecture (just the MLP maybe since DiffNet blocks are like... done?) Any ideas are very welcome.
I already adapted the code in faust_scape_dataset.py and other places in functional_correspondence.py and fmaps_model.py to also calculate SHOT features in the train.pt and test.pt cache files and shortly I'll add support for LPS.
I'll also try to train and run it through GeomFMaps on FAUST_r to see if I can get some features out of it.
So far with SHOT descriptors (used some default settings for SHOT itself, no local reference frame), we don't get anywhere near as good performance as HKS even after 25 epochs of training:
Overall test accuracy: 2.62651e-04 geodesic error: 2.62391e-02
We're training for another 50 epochs currently, but I don't think test geodesic error will improve.
I noticed in the paper you mention SHOT is too complex and wouldn't be beneficial, it seems either you tried it or your speculation was correct!
However, the pre-trained models you provided (especially the one on HKS) yield very good performance,
There are some papers submitted to 2023 confs (CVPR etc.) about invariant or learnable features for correspondence but they haven't published code yet (e.g. clover,vader etc.) (we're interested in FAUST but will take any available code on similar datasets).
We also thought of trying WKS, but I think it will require a lot of adjustments to the code and will make the model harder to train.
Do you have any other features (invariant to rototranslations or even learnable) in mind to suggest?
P.S. BTW, I can make a PR after the work's done if you want (I dunno if the datasets/processed cache .pt files can be published, I'm baffled by all the preprocessing in MATLAB and others in these other codebases).
Hey Nick, TL;DR
Long version: First of all, thanks so much for your (&CMU & Ecole Polotechnique labs) amazing work on DiffusionNet and for open sourcing the code along with pre-trained models!
I'm doing a course project with a friend about improving performance on the SotA (DiffusionNet in this case) FAUST correspondence somehow (bummer: the project deadline is soon), and we had some ideas on integrating other features and/or modifying the architecture (just the MLP maybe since DiffNet blocks are like... done?) Any ideas are very welcome.
I already adapted the code in
faust_scape_dataset.pyand other places infunctional_correspondence.pyandfmaps_model.pyto also calculate SHOT features in thetrain.ptandtest.ptcache files and shortly I'll add support for LPS. I'll also try to train and run it through GeomFMaps on FAUST_r to see if I can get some features out of it.So far with SHOT descriptors (used some default settings for SHOT itself, no local reference frame), we don't get anywhere near as good performance as HKS even after 25 epochs of training:
Overall test accuracy: 2.62651e-04 geodesic error: 2.62391e-02We're training for another 50 epochs currently, but I don't think test geodesic error will improve. I noticed in the paper you mention SHOT is too complex and wouldn't be beneficial, it seems either you tried it or your speculation was correct! However, the pre-trained models you provided (especially the one on HKS) yield very good performance,There are some papers submitted to 2023 confs (CVPR etc.) about invariant or learnable features for correspondence but they haven't published code yet (e.g. clover,vader etc.) (we're interested in FAUST but will take any available code on similar datasets). We also thought of trying WKS, but I think it will require a lot of adjustments to the code and will make the model harder to train. Do you have any other features (invariant to rototranslations or even learnable) in mind to suggest?
P.S. BTW, I can make a PR after the work's done if you want (I dunno if the datasets/processed cache .pt files can be published, I'm baffled by all the preprocessing in MATLAB and others in these other codebases).