visual feature extraction

[shatter947152]

"When using different splits to train a visual feature extractor (e.g., ST-GCN), do you use the entire visible feature test set as the validation set? How can we ensure that, while training solely on the visible feature training set, the accuracy on the visible feature test set reaches around 93% instead of around 82% (as reported in the ST-GCN paper)? If you can see this, please address my concerns. Thank you."

[pha123661]

Hi,

Thanks for your question. We only use seen classes extracted from the entire validation set to test our model. As we use only the seen classes to train and test our model, this makes the problem simpler or harder depending on the selection of the unseen classes. Thus you can't directly compare these two performance numbers.

Thanks, Oscar

[shatter947152]

Hello, first of all, thank you for your response. Perhaps due to the lack of clarity in my initial question, I’ll try to express it more thoroughly this time.

I noticed that in the visual feature files, there is a val.npy file, whose size matches the size of the test set after excluding the unseen classes. I assume you trained the skeleton feature extraction module on all seen classes in the training set and then validated it on all seen classes in the test set. However, I also noticed another file, val_out.npy, which should represent the classification scores of the validation set. Comparing this with the real labels of the validation set’s seen classes in the val_label.npy file, I found that the accuracy is close to 93%. If you trained only on all seen classes in the training set, the model should not reach such high accuracy.

Could you clarify if the skeleton extraction framework is trained solely on all seen class data from the original training set? Or does it also include some seen class data from the original test set?