evaluation

[catherine-qian]

Dear authors,

Thanks for the excellent work and the code contribution! ----- training -------- I have successfully run the train.py script

1. how many epochs you used for training?-> in your paper, you mentioned 'to pretain the model without autoregression for first 7 epochs' and then '5 epochs of training with autoregression', do you use 12 epochs for the final model?
2. do you set an early stop mechanism?

----- evaluation ---------- when I run the evaluation work, it returns the error: no such file or directory : '../dataset/processed/test_inputs/X_test_NaturalTalking_04.npy' ]

I checked the 'test_inputs' folder it only contains 'T_test_Recording_003.npy and X_test_Recording_003.npy'. They are from the 'Training_data' of the GENEA_Challenge_2020_data.

3. Could you please specify which test file you use for the paper?

Thanks a lot for your contribution and looking forward to your reply.

[Svito-zar]

Dear @catherine-qian , I am glad that our work is useful for you! Let me take your questions one by one:

----- training --------

1. I think, we have trained our model for 300 epochs
2. No, we did not use the early stopping

----- evaluation ----------

I will look into that and will fix this issue asap

3. The test files we used in the paper actually correspond to *_test_Recording_0020 and *_test_Recording_022 and the exact time-steps can be found in the following function: https://github.com/Svito-zar/gesticulator/blob/master/gesticulator/model/prediction_saving.py#L96

For more details on that see my reply to the issue https://github.com/Svito-zar/gesticulator/issues/14 about the dataset split.

Hope that helps.

[Svito-zar]

The evaluation.py script should be working now after this commit: https://github.com/Svito-zar/gesticulator/commit/7a14a5d9bc68977fa6bb7926627b718c859f1d0d

[Svito-zar]

@catherine-qian , I realize now that I need to do the renaming properly and carefully to enable full reproduction of the results. Will do my best to make it happen this week.

Meanwhile, as a quick fix, you can download the processed test sequences under this link and put it in the folder with the test_inputs. Let me know if it worked for you.

[Svito-zar]

I have added renaming scripts and updated the code, so now everything should work

[catherine-qian]

Dear svito, thanks for your contribution.

for "python bvh2features.py", could you please let me know what to specify for --bvh_dir and --dest_dir？

[Svito-zar]

If you have done the previous steps according to the README - both of those parameters should point to your directory with the motion files. Normally it would be under dataset/processed_data/Motions. Is it clear now?

Maybe I should clarify that in the instructions ...

[catherine-qian]

thanks for your help. I assume the bvh_dir and the dest_dir both pointing to '../../dataset/raw_data/Motion' and now it works!