Request for training logs and detailed settings

[BotMan-Sz]

Dear authors, Recently I have been studying your paper and code, however, even with your proceesed dataset, I can not repeat your reported results in your paper. Take the HAR dataset as example, there comes the following issues.

1. When I choose the random_init training, the results(77.9) is much better than the scores you reported in Table 2(57.89±5.13). I don't know what random seed you used in your paper, but for 5 seed i tested, they all higher than your report.
2. When I use only 5% of trainining data for supervised trainining, the accuracy and macro F1(>0.89) is much higher than the reported scores Fig.2(MF1<0.55). Below are the results for training with 5% training dataset (with batch size of 8).
3. All thereported results are hard to reproduce.

Hope you could share me your experiment setting and training logs to remove doubts.

[BotMan-Sz]

logs_10_12_2024_07_00_53.log run1_random_init_seed_0_classification_report.xlsx logs_10_12_2024_07_07_03(supervised with 1% HAR dataset).log logs_10_12_2024_07_21_37(supervised with 5% HAR dataset).log run1_supervised_seed_0_classification_report(supervised with 1% HAR dataset).xlsx run1_supervised_seed_0_classification_report(supervised with 5% HAR dataset).xlsx

[BotMan-Sz]

Add this in dataloader/dataloder.py/Load_Dataset() function to obtain certain ratio of tainining samples.

[emadeldeen24]

Hi, thanks for experimenting with our code. Unfortunately, I don't have the logs anymore. But I recall using the same parameters as yours, maybe the dropout was 0.5, but everything is the same. I'm not sure why you can not reproduce the results.

[BotMan-Sz]

Hi, Thanks for your response. As the authors, reproducing your own code should not be difficult, I hope you take a little time to recall your parameter configuration, etc., and run the code again, which is crucial for the reliability of your work.

[BotMan-Sz]

If needed, I can update my projects with logs to github, and i didn't make any changes to the core codes. It seems that even the Random Initialization and supervised results are not reliable, not to say other baselines.