search

wenquanlu / HandRefiner

MIT License
701 stars 30 forks source link

open source #2

Closed WangYuXuan2022 closed 6 months ago

WangYuXuan2022 commented 6 months ago

Excuse me, I really appreciate your work. May I ask when will the code be released?

moclimb commented 6 months ago

All the flowers I waited for have faded！

wenquanlu commented 6 months ago

Hi, I am working on it, (involves some restructuring, making the code more maintainable, and documentations) I cannot give a definite date yet , but it is definitely gonna be released this month.

[I am personally super busy with some other things at the moment, (given it is the graduate studies application season) I am trying my best to get it release early, thanks for your patience!]

Sisimshow commented 6 months ago

Glad to hear you're working on it, I am very looking forward to seeing this :) thank you for all of your work!

moclimb commented 6 months ago

Hi, I am working on it, (involves some restructuring, making the code more maintainable, and documentations) I cannot give a definite date yet , but it is definitely gonna be released this month.

[I am personally super busy with some other things at the moment, (given it is the graduate studies application season) I am trying my best to get it release early, thanks for your patience!]

this month only 3 days left...

wenquanlu commented 6 months ago

Yep finalising it now... should be on time

moclimb commented 6 months ago

Yep finalising it now... should be on time

Thanks god!

Sisimshow commented 6 months ago

Thank you for posting the code :) I made a Reddit post about it, hope that's ok

codelilei commented 6 months ago

Wrote almost the same implementation but with handmesh method, the effect is not satisfactory currently, checking the difference now...

wenquanlu commented 6 months ago

I think fine-tuning also helps

codelilei commented 6 months ago

Wrote almost the same implementation but with handmesh method, the effect is not satisfactory currently, checking the difference now...

I have tried a lot of experiments two weeks ago directly based on the real world dataset which contains different size of images after pre-processing, and the hand ratio looks not so big as the paper shows. Also the training starts finetuning from the sd15-inpainting checkpoint, without using the hand-region-only loss metioned in the paper.

codelilei commented 6 months ago

By the way, does the scaling factor 0.8 to the depth map matter? Seems that it can distinguish the hand part of high depth from the background zeros pixels. Thanks for your reply~

wenquanlu commented 5 months ago

No worries, maybe the scaling factor is not necessary, but I am not too sure... it should not impact the generation result too much.

wenquanlu commented 5 months ago

I have tried a lot of experiments two weeks ago directly based on the real world dataset which contains different size of images after pre-processing, and the hand ratio looks not so big as the paper shows. Also the training starts finetuning from the sd15-inpainting checkpoint, without using the hand-region-only loss metioned in the paper.

Do you mean you have a real world dataset for training? [so you have real world image paired with segmented depth map of hand?]

codelilei commented 5 months ago

I have tried a lot of experiments two weeks ago directly based on the real world dataset which contains different size of images after pre-processing, and the hand ratio looks not so big as the paper shows. Also the training starts finetuning from the sd15-inpainting checkpoint, without using the hand-region-only loss metioned in the paper.

Do you mean you have a real world dataset for training? [so you have real world image paired with segmented depth map of hand?]

yes but not paired data, the mesh is generated by HandMesh method and depthmap is converted from it in the same manner as your code does.