search

YoungSeng / DiffuseStyleGesture

DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models (IJCAI 2023) | The DiffuseStyleGesture+ entry to the GENEA Challenge 2023 (ICMI 2023, Reproducibility Award)
MIT License
147 stars 21 forks source link

the question of smpl model #33

Open pipijiev12 opened 8 months ago

pipijiev12 commented 8 months ago

Suppose I have my own 3D model, how can I convert it to a smpl or smplx parametric model? Let's say this is my 3D model 1 Convert to two files npz and pkl as shown below 2

YoungSeng commented 8 months ago

Thanks for the interest in this work, I would like to know what format your 3D model is in, is it the usual FPX format? As well as DiffuseStyleGesture (ZEGGS dataset) I didn't take into account the conversion between SMPL and SMPLX, you can try DiffuseStyleGesture+ (BEAT dataset) and then use this code of FreeTalker to convert the BVH result to SMPLX format.

pipijiev12 commented 8 months ago

Thanks for the interest in this work, I would like to know what format your 3D model is in, is it the usual FPX format? As well as DiffuseStyleGesture (ZEGGS dataset) I didn't take into account the conversion between SMPL and SMPLX, you can try DiffuseStyleGesture+ (BEAT dataset) and then use this code of FreeTalker to convert the BVH result to SMPLX format.

Another question I have is that now I have used ZEGGS dataset preprocessing, but the data in the mdb file is all 0.

YoungSeng commented 8 months ago

Reference here, my suggestion is to delete the wrongly generated lmdb file (8KB) and re-run the code of that step to generate the lmdb file; also try DiffuseStyleGesture+, that one generates the h5 file, which I think is more optimized compared to generating the lmdb.

pipijiev12 commented 7 months ago

Reference here, my suggestion is to delete the wrongly generated lmdb file (8KB) and re-run the code of that step to generate the lmdb file; also try DiffuseStyleGesture+, that one generates the h5 file, which I think is more optimized compared to generating the lmdb.

I have tried many times to modify the DataPreprocessor function, but the wrong data.mdb and lock.mdb are still generated. 1

YoungSeng commented 7 months ago
  1. I think the code should not need to be modified, just delete the file generated by the bug and run it again to generate the lmdb file; 2. In your case, I suggest trying DSG+, which does not generate the lmdb file but generates the h5 file, thus solving the problem; 3. Continue to try debugging.
pipijiev12 commented 7 months ago
  1. I think the code should not need to be modified, just delete the file generated by the bug and run it again to generate the lmdb file; 2. In your case, I suggest trying DSG+, which does not generate the lmdb file but generates the h5 file, thus solving the problem; 3. Continue to try debugging.

This code builds the mdb data module. Can I rewrite it based on the beat and twh codes?

YoungSeng commented 7 months ago

Of course you can rewrite it, it's just that I think the generated h5 file is smaller and more convenient than the lmdb file.