angerhang
commented
7 years ago First you need to understand that not all the figures can be produced using the code that is in the repo, but the existing code should give you enough idea of how to use different functions.
ans:
- yeah trainDriver is just an example, since we don't have many subjects, we use the k-fold cross-validation result to get the testing error. this is not good but in experiments of a small sample size, this is acceptable because you can't split the data more.
- the data splitting happens in
cross_validatie - yes the coefficients are documented in the report, result section.
- preRMSE and postRMSE are defined in experiments data source section.
- No I meant what I wrote. All subjects's performance improved after training and all of them have negative MLS. The smaller MLS is, the more they have improved due to training.
- Lolol then this is a mistake in the writing. There are 77 training samples. I will correct that in the report later. Thanks for pointing this out.
liutianlin0121
More questions:
It seems that the script
trainDriver.monly reconstructs the training errors. Would you extend this script such that the testing errors, as shown in Table 1 of the report, could be reconstructed?In fact, I don’t see where are the testing samples in the repo. Correct me if I am wrong, but it seems to me that for each model, say, the model OH vs YH, you use up all samples under this model to train the ESN. Would you specify how did you divide training and testing sets in the report for each model? Note “testing sets” in k-fold cross-validation aren’t really testing sets. The genuine testing set must be the datasets such that the ESN never encounter in the whole training phase.
Running
trainDriver.m,I see the training error is approximately 0.18, which matches what Table 2 claims in your report. However, if I use other datasets, say, the first and the third dataset, the training error is approximately 0.3, which is far from what the Table 2 in reports claimed. I think probably this is due to the parameters— one has to choose different sets of parameters for different datasets. Would you specify what parameters should be used for first, second, and third dataset respectively, such that the results in table 2 could be reconstructed?Some questions I asked in the last week are not yet addressed in the report. I copy the questions that still puzzle me here.
PreRMSE and PostRMSE are defined as the motor performance before and after the motor learning training. But what do PreRMSE and PostRMSE exactly refer? Perhaps the RMSE of the first trial and that of the eighth trial in the modulation task experiment?
The report states: “For the relative MLS, the smaller the value is, the more one’s motor skills can improve after training.” Perhaps instead of smaller you mean larger. Indeed, suppose one has MLS of 0, which is small enough. But it is exactly the case that PreRMSE = PostRMSE, implying that she has no improvement at all.
In Table 1, there are 28 + 23 + 24 = 75 subjects in total. But in page 14 you mentioned that there are 77 samples: Finally, we get rid of the relative MLS outliers which leave us with 77 training samples. Why there are 2 subjects missing? This problem is non-trivial, as once the code is released, it may leave the reader the impression that you are using more training samples than you claimed. For instance, for model b (OL vs YH), there are there are 53 names in that dataset
thirdGroup.mat. So you are training the ESN with 53 samples. But according to the table 1 in the report, one only assumes 28 + 23 = 51 samples are used.Thanks!