Closed H0R5E closed 1 year ago
Hi Ryan,
I implemented your algorithm, but I'm not sure it's behaving the way you expected it to do. The diagram below shows a simulation with the tolerance set to 1e-6
.
I don't think this is converging as you were expecting, but I'm not really surprised because of the random numbers.
I'm also not sure what the tolerance is telling you. I set the tolerance to 1e-4
and ran the algorithm 10 times. The range on the results was 69039.8
Watts, but 1e-4 * max(power)
is 395.2
Watts, so the tolerance is kind of misleading.
How do you want to proceed?
I've added a metric which calculates the sum of the standard errors over the sum of the mean (per frequency component). It produces results as such:
I'll do the reverse and set a limit on the error and show how many iterations are required.
OK, so here is the same situation recording the number of iterations required:
So, I really think the question is whether the user wants to set a particular accuracy and wait for the realisations, or choose a number of realisations and suffer the inaccuracy. I suggest we offer both options.
Just a note that this would need to be updated if #170 is merged first and visa-versa.
As part of the testing process I noticed that when summing the errors the results were somewhat better than expected. Turns out, that's because summing the standard error is not the correct approach and taking the 2norm is the correct way to go. I've implemented that and the results are now more consistent.
Add option when reducing error for max samples - return error.
Done.
@ryancoe
Description
This commit uses the central value theorem to facilitate calculation of the error on the mean results of the PS controller, using the standard error from a normal distribution.
Two new functions are added to the WecOptTool.math package called standardErrorMeasure and standardErrorReduce, the first of which measures the standard error after a given number of samples and the second of which keeps sampling until a desired level of error is found.
An example script to show how the standard error varies with the stroke of the RM3 device using the PS controller is added to the RM3 example directory called PSError.m. The results of this simulation are shown here:
Fixes #56
Checklist:
[ ] Ifexamples/RM3/optimization.m
has been modified, the content / line numbers indocs/user/optimization.rst
are still valid or have been fixed[ ] Add something to the documentation?-> Do as part of #153