Open ziyuwwang opened 1 year ago
Hello @ziyuwwang,
Thanks for your interest in our work! Also, thanks for your patience, I was on leave for the past 3 weeks.
Here are the answers to your questions:
pc_assembly
. However, for MECCANO, there were 2-3 videos where the subject is unable to follow the steps provided and are unable to finish assembling the toy. Due to this, we have not annotated those exceptional videos.Let me know if there are any other concerns!
Thanks for your response! Apart from the previous questions, I have some other problems:
Sorry, another question: I found the data transformation when evaluating models is also confusing. When training a model, input data is read from h5 files and then divided by 127.5 and subtracted by 1. This makes the value range of the input data (-1, 1). But when performing procedure learning (evaluating), the input data is read from h5 files and then converted into PIL Image formation. A normalization is followed and the values are still divided by 127.5 and subtracted by 1. These operations make the values become negative numbers with very small absolute values. I am not sure whether it is right. Can you check this for me?
@ziyuwwang Thanks for your questions! Here are the answers:
Do you use the average metrics or the overall metrics in your paper? I found these two are quite different.
We use the average metrics for evaluating the models when we are not comparing with the previous works (Table 3 onwards). However, to make sure there is a fair comparison with the previous works, as mentioned in Section 5.2 in the paper, we use the overall metric when comparing in Table 2.
Do you set the hyperparameter "KMEANS_NUM_CLUSTERS" to be 7 for all the datasets? Is this hyperparameter the same as the hyperparameter "K" in your paper?
Yes, you're correct. KMEANS_NUM_CLUSTERS
is 7 for all the dataset (as determined from Table 6 in the paper). We change this value when performing the hyper-parameter tuning (reported in Table 6).
It would be better if you can provide your configuration file for each dataset.
Thanks for the suggestion, I'll upload these files and notify you.
Doubts on data transformation.
Thanks for pointing this out, and apologies for the confusion here. In our experiments, we have used transforms = None
here: https://github.com/Sid2697/EgoProceL-egocentric-procedure-learning/blob/01fa3df0bb2af718030ca22bfcea7bfa26110080/RepLearn/TCC/procedure_learning.py#L95
This makes the data pre-processing similar to what is performed during the training. The transformations are only applied by this method: https://github.com/Sid2697/EgoProceL-egocentric-procedure-learning/blob/01fa3df0bb2af718030ca22bfcea7bfa26110080/RepLearn/TCC/utils.py#L44
Let me know if there are further questions!
@Sid2697 Thank you for taking the time to answer my questions. But I really have many questions yet:
Hope you can resolve my questions. Thanks!
Hello @ziyuwwang , thanks for your response! Here are my answers:
I wish to get deeper into the issue in point 3, however, I'm currently occupied with my PhD projects. Due to this, it will take me a while to have a look at the code and dig deeper. That is why I recommend you to please reach out to the original repository for a faster resolution.
@ziyuwwang this paper might be of interest to you: https://arxiv.org/pdf/2301.00794.pdf
Hi Sid, Thanks for your wonderful work! When I ran the code, I met a few questions:
Can you check these issues and provide complete your datasets so that I can smoothly follow your work? Thanks a million!