gfmoore / esci-dances

A JavaScript implementation of the ESCI software simulator used in Introduction to The New Stastistics (by Geoff Cumming and Robert Calin-Jageman)
https://thenewstatistics.com/esci-js/
Other
0 stars 0 forks source link

The logic behind what should be seen or not when checkboxes checked unchecked #17

Closed gfmoore closed 4 years ago

gfmoore commented 4 years ago

Okay, I'm losing the plot here. I'm struggling to come up with a consistent logic as to what should be displayed (on dropping means) when various checkboxes are checked or unchecked.

My code is starting to resemble spaghetti again! :)

The checkboxes are: DataPoints SampleMeans Dropping Means

Mean Heap CIs Capture of mu

Displaying: Panel 4: Number of samples Latest sample M and s MoE population MoE sample

Panel 5: Mean heap M and SE Number of means in the heap
(CurveSE is obvious I think)

Panel 7: Number capturing mu Samples taken (duplicating panel 4 Percent capturing mu

Any direction gratefully received. (And I think you've tried to steer me right on a number of occasions before).

I have a ClearAll function that resets all buffers and variables. (The clear button calls it). But I can clear or display/not display at a deeper level. This is what needs tidying up.

I have a buffer that contains enough information on every sample taken for recalculating capture rates.

Finally, what do you want to do about the visible heap that grows too big? It keeps generating "blobs" even if it cannot display them in the visible part of the display? Memory hog. Again you have made comments on this!

gdcumming commented 4 years ago

When any control (button, slider, checkbox, radio button) is clicked, one or more of these should happen: S (Stop)--stop the sampling, if this is running C (Clear)--clear means and CIs in lower panel, reset counts to zero, i.e. be ready to start a new run A (Action)--carry out the action of that particular control

My suggestions below don't capture all the finer dependencies, for example if, in Panel 2, pop is clicked off, the other two checkboxes are also cleared. If SD is then clicked on the box switches on, but nothing is displayed until pop is clicked on.

Panel 1 Any change to mu or sigma: SCA Any change to pop shape or skew value: SCA i.e. don't automatically restart sampling if any of those are changed while sampling is running

Panel 2 Any change: A i.e. change display in upper panel any time, without any change to what may be happening below

Panel 3 Clear button: SA (same as SCA in this case) Take sample: SA (don't automatically restart running sampling) Run-Stop: A Slider: A (can adjust speed during a run)

Panel 4 N dropdown: SCA Data points: A Sample means: SCA Dropping means: SCA

Panel 5 Mean heap: SCA (as at present) Sampling distribution curve: A (as at present) SE lines: A (as at present) But show these only when sampling distribution curve is visible. +/- Moe lines: A (as at present) But these don't update in some situations, e.g. change mu or sigma, or pop shape

Panel 6 C dropdown: SA, then C if a sample is taken or a run is started. I.e. change C% and sampling stops, and we see all visible CIs change in length, some change in colour maybe. Can change C% further. But then we need an automatic clear when either Take sample or Run button is pressed. This ensures that no single run includes CIs of different C values. It's great that number capturing and capture % update, I think correctly, as C% is changed (with a stationary display of means and CIs, maybe mean heap. But at present it's possible to get red in the mean heap that relates to a previous value of C. Best to have a Clear before starting another run.

CIs checkbox: SA, then C if a sample is taken or a run started. (See earlier notes about turning off capture if CIs turned off. So auto-off for 'Capture of mu' checkbox if CIs are turned off.)

Known/unknown radio buttons: If selection changed: SA (can switch back and forth, with sampling stopped, and see what happens to capture %, red in mean heap, lengths of CIs, etc. Then can restart the run, perhaps with a different selection of known/unknown. This seems to me a bit dangerous, but, given you have such a great buffering system for recording info on all samples in a run, it seems to work reliably so far as I can see. And can be explained to students I think. What happens with very high numbers of samples in a run? Do we have to set some limit? Or will it just very gradually slow down--at present it runs fast for a very long time. BTW, in answer to your question about the mean heap growing out of sight--I don't think that's a problem. It looks natural, if anyone is silly enough just to go on running samples for ages.

Panel 7 mu line: SCA and auto-clear 'Capture of mu' checkbox Capture of mu: Requires mu line visible and CIs on for this checkbox to come on when clicked. Click this on: SCA. Click it off: SCA.

Phew--I hope that's consistent and makes sense--and is not a terrible pain to build. A wonderful set of educational possibilities!

gfmoore commented 4 years ago

Panel 1 Any change to mu or sigma: SCA Any change to pop shape or skew value: SCA i.e. don't automatically restart sampling if any of those are changed while sampling is running

I think I've got this panel sorted! (oh and indent for markdown!!)

Panel 2 Any change: A i.e. change display in upper panel any time, without any change to what may be happening below

I don't understand? Upper panel = panel 1? Left alone for now

Panel 3 Clear button: SA (same as SCA in this case) Take sample: SA (don't automatically restart running sampling) Run-Stop: A Slider: A (can adjust speed during a run)

Done

Panel 4 N dropdown: SCA Data points: A Sample means: SCA Dropping means: SCA

Done

Panel 5 Mean heap: SCA (as at present) Sampling distribution curve: A (as at present)

Left as is

SE lines: A (as at present) But show these only when sampling distribution curve is visible.

???? If they don't show when mean heap is off, should the SE Lines checkbox be cleared when the mean heap check box is cleared?

+/- Moe lines: A (as at present) But these don't update in some situations, e.g. change mu or sigma, or pop shape

Left as is

Panel 6 - next comment

Panel 7 mu line: SCA and auto-clear 'Capture of mu' checkbox

done

Capture of mu: Requires mu line visible and CIs on for this checkbox to come on when clicked.

Click this on: SCA. Click it off: SCA.

Uhmm. I quite like it that this toggles the display of capture not capture when muline and CIs on. Do you really want it to clear when cleared? The clear button can clear this if you want to start afresh. However, as you wish :) Let me know.

I also added to the code to ensure that the +/- MoE lines around mu stayed on top - though this is yet more processing each time a mean joins the heap, and also the mu line similalry.

Are you happy with the colours of these lines as they seem to get a bit washed out in the heap - could be black?

gfmoore commented 4 years ago

Panel 6 C dropdown: SA, then C if a sample is taken or a run is started. I.e. change C% and sampling stops, and we see all visible CIs change in length, some change in colour maybe. Can change C% further. But then we need an automatic clear when either Take sample or Run button is pressed. This ensures that no single run includes CIs of different C values. It's great that number capturing and capture % update, I think correctly, as C% is changed (with a stationary display of means and CIs, maybe mean heap. But at present it's possible to get red in the mean heap that relates to a previous value of C. Best to have a Clear before starting another run.

Yes, hadn't noted this before. So change on CI => SCA

CIs checkbox: SA, then C if a sample is taken or a run started. (See earlier notes about turning off capture if CIs turned off. So auto-off for 'Capture of mu' checkbox if CIs are turned off.)

I think this relates to another issue, I think it is now correct.

gfmoore commented 4 years ago

Panel 6 Known/unknown radio buttons: If selection changed: SA (can switch back and forth, with sampling stopped, and see what happens to capture %, red in mean heap, lengths of CIs, etc. Then can restart the run, perhaps with a different selection of known/unknown. This seems to me a bit dangerous, but, given you have such a great buffering system for recording info on all samples in a run, it seems to work reliably so far as I can see. And can be explained to students I think. What happens with very high numbers of samples in a run? Do we have to set some limit? Or will it just very gradually slow down--at present it runs fast for a very long time. BTW, in answer to your question about the mean heap growing out of sight--I don't think that's a problem. It looks natural, if anyone is silly enough just to go on running samples for ages.

So shall we leave as is, again I like the dynamic nature of it. When the sigma know unknown is changed the program recalculates everything, so should be okay. I guess if there are tens of thousands of sample taken then there might be a delay as it recalculates and recolours, but how often is that going to happen? (Probably a lot now that I've said it! lol)

gfmoore commented 4 years ago

This has been extremely useful. I think it is quite amazing how my thinking was often quite different from how you viewed things.

I appreciate now that you have seen ESCI as a pedagogical process, I have seen it as "features" to be turned on and off. Good lesson to be learnt for future applications.

Thankyou for your efforts here.

We should do this before embarking on a next app. Probably make it an "issue" (closed?) for constant reference.

gdcumming commented 4 years ago

C causes the fill under the pop to be re-randomised. That doesn't matter, unless it causes too much delay. Which possibly it does?

Clear checkboxes in Panel 4. C. Click 'Sample means' on, then take a sample (or do Run then Stop) and see one single mean, without CI, even if CI had been clicked on. Take another sample and all is well--see mean and CI.

Panel 5, still to do: SE lines: A (as at present) But show these only when sampling distribution curve is visible. (Yes, I think you are right: when sdc turned off, SE checkbox should automatically clear.) +/- Moe lines: A (as at present) But these don't update in some situations, e.g. change mu or sigma, or pop shape

Panel 6, still to do. C dropdown: SA, then C if a sample is taken or a run is started. I.e. change C% and sampling stops, and we see all visible CIs change in length, some change in colour maybe. Can change C% further. But then we need an automatic clear when either Take sample or Run button is pressed. This ensures that no single run includes CIs of different C values. It's great that number capturing and capture % update, I think correctly, as C% is changed (with a stationary display of means and CIs, maybe mean heap. But at present it's possible to get red in the mean heap that relates to a previous value of C. Best to have a Clear before starting another run.

    GM comment: Yes, hadn't noted this before. So change on CI => SCA

No! I'm asking that on change to C, we get SA. So we can see the display of dropping means+CIs, now stopped, with the new CI lengths. Might change C a few more times and note the CI length changes. But if another sample is taken, or Run is clicked, then we should get a Clear. Thus avoiding any confusion about cumulating capture counts from CIs with different C% lengths.

It's good to have mu line and +/-MoE lines in front. I think their colour is ok.

The big questions: What should happen when 'capture of mu', or 'CIs', are toggled on or off. I had always felt that we needed SA, then C on further sampling. But I take your point that, once we're used to how things work, it can be good to change such things and simply go on sampling. This is only possible, without weirdness, because you keep so much info about the current run that we can see accurate capture %ages and counts, even after various changes. I'm not totally convinced--see dropping means, without CIs, you counting of capture of by those invisible CIs still goes on!

Let's leave this as it is for now, and I'll consult Bob.

THIS IS THE OLD COMMENT, NOTHING ABOVE CHANGED. I WANT TO MAKE A NEW COMMENT, BUT TYPED SOMETHING, AND i CAN'T FIND A WAY TO CANCEL, AND START A NEW COMMENT

gfmoore commented 4 years ago

C causes the fill under the pop to be re-randomised. That doesn't matter, unless it causes too much delay. Which possibly it does?

Clear checkboxes in Panel 4. C. Click 'Sample means' on, then take a sample (or do Run then Stop) and see one single mean, without CI, even if CI had been clicked on. Take another sample and all is well--see mean and CI.

I think this is fixed, maybe, or maybe introduces yet another bug...

:)

gdcumming commented 4 years ago

0.3.48

I have copied across from last night's partial comment, deleted that, and edited as below:

Bob said in an email that maybe more rather than less clearing would be good. I'm going to suggest a middle position. Could we try this, and see how it feels and how it can be used for explanations in Chaps 4 and 5, which I'll be working on soon.

S, C, and A as before. Add 'F' meaning don't clear display but set a flag so that when 'Take sample' or 'Run' is next clicked, 'C' occurs before the new sampling. That's so we can observe the effect of a change, e.g. change the C% value of the CIs on screen. We see them get shorter or longer, and change colour accordingly, but if we start sampling a Clear ensures that we're not possibly confused into thinking that the % capture comes from some combination of short and long CIs, given by the original then the new values of C%.

Panel 1 (as before) Any change to mu or sigma: SCA Any change to pop shape or skew value: SCA i.e. don't automatically restart sampling if any of those are changed while sampling is running

Panel 2 (as before) Any change: A i.e. change display in upper panel any time, without any change to what may be happening below

Panel 3 (as before) Clear button: SA (same as SCA in this case) Take sample: SA (don't automatically restart running sampling) Run-Stop: A Slider: A (can adjust speed during a run)

Panel 4 (as before) N dropdown: SCA Data points: A Sample means: SCA Dropping means: SCA

Panel 5 (with two changes) Mean heap: A -- a new idea: the focus is on the dances, i.e. the dropping means (possibly with CIs). We may elect to collect these into a heap. It makes sense to be able to show or hide the heap, without interrupting or influencing the dropping means. Sampling distribution curve: A (as at present). If SDC is turned off, SE lines checkbox should be cleared, and can't be turned on. SE lines: A, but turning this on should only be possible when SDC is on. +/- Moe lines: A (as at present) But these don't update in some situations, e.g. change mu or sigma, or pop shape (I'm going to make that a new issue)

Panel 6 (now with added F!) C dropdown: SAF I.e. stop, change CI lengths, maybe colours and capture counts and %, accordingly, and set F to prepare for C when a button is clicked for further sampling--which amongst other things will reset the capture count to zero before the sampling starts.

CIs checkbox:

  1. If 'Capture of mu' is OFF, then A (for turning CIs either on or off)
  2. If 'Capture of mu' is ON, then ---turning CIs off: simply A, meaning that red/green means can continue to fall, possibly onto heap ---turning CIs on: also simply A, meaning that the red/green falling means now once again have CIs attached. (Because--see below--we require CIs to be on before 'Capture of mu' can be turned on, so turning CIs on can't initiate a run, but it can continue a previously started run.)

Known/unknown radio buttons: If selection changed: A --as at present, whether sampling is running or not. This seems to me a bit dangerous, but, given you have such a great buffering system for recording info on all samples in a run, it seems to work reliably so far as I can see. And can be explained to students I think.

Panel 7 mu line: ---If 'Capture of mu' is off, simply A. I.e. can show or hide mu line while running sampling, when capture is off. ---If 'Capture of mu' is on, then mu line is being displayed. If mu line clicked off, then 'Capture of mu' should be forced off, as at present. (Unlike at present, if mu line turned on again, can't turn 'Capture of mu' back on again and see the old counts. Need to start a new run. As below.)

Capture of mu: Require mu line visible and CIs on for this checkbox to come on when clicked. I.e. unless both mu line and CIs are on, the 'Capture of mu' checkbox can't be turned on. (It is usually clear in this situation, but may be still on if 'CIs' checkbox has been cleared during a run, as above.) In other words, 'Capture of mu' can't be clicked from off to on unless both mu line and CIs are on.

If both mu line and CIs are visible, then 'Capture of mu' can be turned on, but causes SCA. So a new run must be started. (So we can't ever cumulate capture counts over samples, some taken with Capture of mu on and some with it off.)


I hope all the above makes sense, and gives a logic that gives rich yet comprehensible possibilities for pedagogy. Let's try it anyway.

gdcumming commented 4 years ago

0.3.48

I made a comment above about +/- MoE lines not updating and that I'd make a new issue. I think you have fixed this, thanks.

gfmoore commented 4 years ago

Panel 5 (with two changes) Mean heap: A -- a new idea: the focus is on the dances, i.e. the dropping means (possibly with CIs). We may elect to collect these into a heap. It makes sense to be able to show or hide the heap, without interrupting or influencing the dropping means.

If the heap is on we see the growing heap. If the heap is turned off I can hide the heap, but does it continue to grow unseen? If the heap is turned back on, depends on the answer to the above I suppose.

My current logic is now based on the show heap flag, if it is off it doesn't add to heap it just drops to the bottom. If you wanted the heap to keep growing then you would have the situation where dropping means would disappear where the heap grows. If the heap is turned back on it starts adding again to the heap. I'm trying to remember what statistics are affected by adding to the heap. If means keep being produced maybe the counts get changed. I'll have to trace the logic. So I'll wait till you decide.

gfmoore commented 4 years ago

Sampling distribution curve: A (as at present). If SDC is turned off, SE lines checkbox should be cleared, and can't be turned on.

SE lines: A, but turning this on should only be possible when SDC is on.

implemented

gfmoore commented 4 years ago

+/- Moe lines: A (as at present) But these don't update in some situations, e.g. change mu or sigma, or pop shape (I'm going to make that a new issue)

I think this is now fixed?

oops you said this.

gdcumming commented 4 years ago

0.3.55

SDC and SE lines fix: thanks. Thanks also for +/-MoE lines updating.

Show/hide mean heap: Ah, I see. Sorry, I hadn't thought that through sufficiently. Can get very tangled.

For simplicity, and to get this done, let's have SCA, when mean heap is either clicked on, or clicked off. That should also keep the counts of means in the heap and samples taken in sensible alignment.

BTW, if SE lines are on, we see them if SDC also on. Turning off the mean heap should not hide the SE lines, as at present (if SDC is on).

gfmoore commented 4 years ago

Show/hide mean heap: Ah, I see. Sorry, I hadn't thought that through sufficiently. Can get very tangled.

reverted. Not a problem if you want me to go back, I/we just need to figure out if anything needs to get stopped from updating when heap not visible.

BTW, if SE lines are on, we see them if SDC also on. Turning off the mean heap should not hide the SE lines, as at present (if SDC is on).

Turning off the mean heap turns off the sampling distribution curve, so turn off the SE lines. Can't have SE lines or SDC if no heap? Is this right?

gfmoore commented 4 years ago

Panel 6 (now with added F!) C dropdown: SAF I.e. stop, change CI lengths, maybe colours and capture counts and %, accordingly, and set F to prepare for C when a button is clicked for further sampling--which amongst other things will reset the capture count to zero before the sampling starts.

I'm assuming these are separate (at least till I work through this one.) :) So I've implemented this, but not as simple as you might have thought. I had to add additional information to the heap bubbles so that when recoloured on C change all the MoE could be recalculated. This may have some consequences that I've not thought about such as calculations. I think I've done it right and it looks okay, but I need the expert QA inspector. Do you happen to know of one? :)

gfmoore commented 4 years ago

CIs checkbox:

If 'Capture of mu' is OFF, then A (for turning CIs either on or off)

If 'Capture of mu' is ON, then ---turning CIs off: simply A, meaning that red/green means can continue to fall, possibly onto heap

Is this a change now? Back to what I suggested/had?

---turning CIs on: also simply A, meaning that the red/green falling means now once again have CIs attached. (Because--see below-

Okay, I've re-instated this code. Good job I kept it.

-we require CIs to be on before 'Capture of mu' can be turned on, so turning CIs on can't initiate a run, but it can continue a previously started run.)

Ah come on! Sorry guys, but this getting beyond my ability to control, You need a state machine to handle all these combinations. I'm sorry but I spend hours on every change you want, my wife is starting to complain and I'm not getting any closer to completion!!! Sorry, but ...

Can we have a break now please from any further requests. I can fix the existing bugs, but unless it's something really important enough is enough.

Sorry:(

gfmoore commented 4 years ago

I'm closing this issue even though some things are not implemented perhaps. If there is a critical issue then we can open a new issue and fix things one at a time. Thanks for your understanding :)