search

HavenFeng / ArtEq

Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance (ICCV2023)
https://arteq.is.tue.mpg.de/
Other
46 stars 3 forks source link

numerical instabilities & occlusion #2

Open mockup-partner opened 11 months ago

mockup-partner commented 11 months ago

Hi

Thanks for your amazing work. Arteq is able to really provide quickly better performance compared to other baselines. I had two questions related to your repo.

I noticed that in your training script you manually set to zero any gradient that would be nan and indeed some nans occur time to time during training. Do you have any ideas where they could from ?

Also on a side note, after some experiment and training, I observed that arteq seems to work even on partial point clouds (for example front view like depth map from stereo images) but struggles more when it comes to a complete temporary occlusion for example an obstacle hiding the whole elbows. Do you have any suggestions at training time as how to improve its performance in this kind of situation ? For example would conditioning pose estimation to its parent but also grand-parent or even child make sense ?

Thanks in advance and again congratulation for your great work.

mathXin112 commented 11 months ago

@mockup-partner Hello, I would like to ask, when you are training the network, what kind of training resources do you use

mockup-partner commented 10 months ago

Hi @mathXin112, if your question is about the dataset, I am using AMASS and if it's about the compute power, I simply use a RTX 3070 on cuda 12.3. Hope it answers your question.

mockup-partner commented 10 months ago

Hi @mathXin112 , if your question is about the dataset, I am using AMASS. And if it's about the compute power, I am simply using my RTX3070 on cuda 12.3.

Hope it answers your question.

mathXin112 commented 10 months ago

----------------The message has been received，best wishes!您好，邮件已经收到，祝好。

DavidTu21 commented 6 months ago

Hi @mockup-partner. I hope you are doing well! I am also interested in the model's performance on partial input human point cloud, but after some testing or inferencing, the model does not produce good results on partial point cloud (please find the attached inference results for partial input and output, and this model is trained on the default setting). May I ask did you train the model on any other data? Thank you in advance for your time!

Input: image Output: image