Towards a spec

[gdcumming]

You have made some great steps, I'm only beginning to catch up.

Basic idea is to recreate the 'Precision two' and 'Precision paired' sheets of ESCI intro (for ITNS), with hopefully not too many changes. Yesterday I could see what you've done, including black curve with N values attached, for both tabs. Great. Today I can't access http://www.esci-precision.thenewstatistics.com/#tab-1

I get message "server IP address could not be found"

These first spec comments are based on my memory of what you have done, plus some decisions Bob and I made on zoom yesterday.

Call this component 'precision for planning' all through. Both words matter.

Two tabs for the two cases is good. Call these 'Two Independent Groups' and 'Paired Design'

In general, follow the font sizes and spacing of 'distributions', which also has two tabs. I.e. more spacing than the initial version of 'precision'.

Looking back to ESCI for UTNS (the file ESCI chapters 10-13), there were three precision for planning pages, including one for single samples. Also black z curves were plotted. We've decided not to ask for a single sample tab, and not to include the z case. I.e. generally follow ESCI intro, not the earlier ESCI for UTNS.

We would like the MoE sampling distribution curve always to be displayed--the small curve at the bottom. I'll make a separate issue about how to calculate this. I'm still trying to convince myself that what I did in ESCI is correct :-)

At the Paired Design tab, let's have separate panels for the top two items: Population correlation, and Target MoE.

On both curves (on average, and with assurance) have dots and N values at intervals of 0.05 of target MoE, as you have done. The numbers can be turned off. There are also (slightly larger, and/or bold) numbers where the cursor intersects a line, and these numbers can't be turned off.

At both tabs, let's have the Target MoE axis run from 0 to 2.0, with the slider allowing choice of values between 0.05 and 2.0, in steps of 0.005, i.e. much finer than in ESCI. (Let's have a look at how that looks and feels. It may be better to have the axis run to a smaller upper limit, as in ESCI intro (1.5). Also, it may be ok to let the slider go a little below 0.05.)

For Paired Design, let's have the rho slider run from 0 to .99, in steps of .005, i.e. shorter range and finer steps than in ESCI intro.

On opening (arrival from Main Menu), each tab is set to 'truncate display' at 0.25 and Target MoE = 0.4. 'Display all N values' is ON. Paired Design tab is set to rho = .70. Only the black curve ('on average') is displayed.

State of a tab is saved, so that if switch to the other tab, then return, see the first tab just as we'd left it. (This is unlike distributions, in which returning to a tab always finds it back in its default state.) Let's see how this works--I'm thinking of making it easy to switch back and forth to see how N values compare for the two designs.

[gfmoore]

Call this component 'precision for planning' all through. Both words matter.

Done for all visible headings, but code stays at precision* - too many things to change and no-one sees it.

Two tabs for the two cases is good. Call these 'Two Independent Groups' and 'Paired Design'

Done

In general, follow the font sizes and spacing of 'distributions', which also has two tabs. I.e. more spacing than the initial version of 'precision'.

Done (as far as I can, for instance I used a different slider on distributions so slight difference).

Looking back to ESCI for UTNS (the file ESCI chapters 10-13), there were three precision for planning pages, including one for single samples. Also black z curves were plotted. We've decided not to ask for a single sample tab, and not to include the z case. I.e. generally follow ESCI intro, not the earlier ESCI for UTNS.

I don't think I need to do anything here do I?

[gfmoore]

At the Paired Design tab, let's have separate panels for the top two items: Population correlation, and Target MoE.

I've done this, but swapped the order to at least try and match tab 1. Though this doesn't match for later panels :( . Can the Population correlation go at the bottom? What colour could we use as it currently nearly matches panel 5 (I'm looking at Paired Designs tab. It's not hard to tweak this to get the right aesthetic.

[gfmoore]

At both tabs, let's have the Target MoE axis run from 0 to 2.0, with the slider allowing choice of values between 0.05 and 2.0, in steps of 0.005, i.e. much finer than in ESCI. (Let's have a look at how that looks and feels. It may be better to have the axis run to a smaller upper limit, as in ESCI intro (1.5). )

How does this affect the Truncated display? This currently has a minimum of 0.1. I've changed it to 0.05

Let's have a look at how that looks and feels

Do you want me to calculate these intermediate points in the array. How do I display them? I think we need to think a bit more on this otherwise I am going to be changing code and then changing it back???

Problem with Target MoE running to 2 is that the curve is an asymptote to the bottom axis!!! I think that's why you had 1.5 as the max? I've left the scale running to 2, but not changed the code for generating lines so you can see it.

Also, it may be ok to let the slider go a little below 0.05

What do you mean by this?

[gfmoore]

For Paired Design, let's have the rho slider run from 0 to .99, in steps of .005, i.e. shorter range and finer steps than in ESCI intro.

What's ESCI Intro??

Done.

[gfmoore]

There's a problem with the calcs in that after target moe of 1.55 you can get a negative N when calculating the critical value

cv = Math.abs(jStat.studentt.inv( alpha/2, N - 1 ));

You can see this when the points jump to the top of the display.

I suspect you ran into these issues when you did ESCI and that's why you had various limits?

I'm going to wait on some kind of resolution to these issues.

Having an N of 1 doesn't seem to make much sense to me either?

[gfmoore]

On opening (arrival from Main Menu), each tab is set to 'truncate display' at 0.25 and Target MoE = 0.4. 'Display all N values' is ON. Paired Design tab is set to rho = .70. Only the black curve ('on average') is displayed.

done, I think.

[gfmoore]

State of a tab is saved, so that if switch to the other tab, then return, see the first tab just as we'd left it. (This is unlike distributions, in which returning to a tab always finds it back in its default state.) Let's see how this works--I'm thinking of making it easy to switch back and forth to see how N values compare for the two designs.

What about TargetMoE - at the moment I use the same value for use in both tabs?

[gfmoore]

Ok, I'll upload what I have so far. BTW the assurance curve is NOT correct, I just added a fixed value to N to get something to work with. I haven't worked through the calcs in the chapter(s) yet.

[gdcumming]

0.0.3

All of a sudden I can't access precision-for-planning. I guess you and/or Bob are tweaking something.

So I can't make sensible comments about much of the above. But I do recall that this version looks good in lots of ways. I agree that the limits on the horiz axis, and step size needs some thought. I have it on the list to ask Bob. He asked for the .005 steps (in target MoE, and rho in the paired design case). Probably needlessly fine-grained. .01 steps probably ok.

Arbitrary values for the assurance curve--aha, that supersedes my last comment on issue #1.

'ESCI intro' is ESCI for ITNS. Of relevance here: 'ESCI intro chapters 10-16'. One of the files I think I sent. The other is 'ESCI chapters 10-13', which is the earlier version, for UTNS, with more detailed and complex precision stuff. (Three sheets, not two; H axis goes to 2.0; includes a z curve.) We mainly want the simpler versions of the two precision sheets in ESCI intro.

Agreed that N=1 makes no sense. In both ESCIs I set N = 3 as the min, whether two groups each of N=3, or N=3 in paired design. I'm tempted to set N=3 again, tho' I guess that we could allow N=2, but make that a minimum. On the list to check with Bob.

I don't know what chi sq function you use. I did discover that in ESCI (both versions) I used the Excel function CHIINV(prob, df) which is now regarded as superseded, tho' still supported. The modern equivalent is CHISQ.INV(1-prob,df), just to be maximally confusing.

So we want, for 99% assurance, e.g. CHIINV(.01,df) == CHISQ.INV(.99,df), to give the critical value of chi-sq with only 1% of the area to the right of this crit value.

[gdcumming]

0.0.3 Aha, I have it again, thanks.

Your messages above, from the top.

"Call this component..." Fine

"At the Paired Design tab..." Swapping those two seems fine to me. The green for rho matches the same in 'dance r' -- if anyone every notices! I think rho is so central that it needs to be high up. Is there some other colour you could easily use for 'Truncate display' in both tabs? Maybe a buff or light brown you have used elsewhere? Just whatever you think looks ok.

"At both tabs..." Some of my comments above are relevant. Not sure I can say much more. Further consultation with Bob will help. It's not out of the question that we decide to go up to 1.5 (or something) rather than 2. In ESCI intro the tables of values go up to 2.0, but only displayed to 1.5. A late decision I think, partly guided by what screen sizes folks were likely to have, partly by the right tail with lots of stupidly low N didn't appeal. But still, I'm not sure.

"For Paired Design..." Answered above.

"There's a problem with the calcs..." Not sure I follow exactly. In my calcs I set N=3 as the minimum at every stage. Perhaps that's a not unreasonable kludge that can avoid the problem? Or even N=2?

"On opening..." All ok except target MoE = 0.40 is the suggestion, not the current 0.25.

"State of the tab..." Interesting idea to tie the target MoE sliders together. Not sure which I prefer. I can see pros and cons. Leave for now. It's on the Bob consult list.

"Ok I'll upload..." Assurance--comments elsewhere.

[gfmoore]

https://www.esci-precision.thenewstatistics.com/#tab-1  just checked, seems ok my end?

regards,

Gordon Moore gm@gordonmoore.co.uk

On 22/10/2020 05:29:14, Geoff Cumming notifications@github.com wrote: 0.0.3 All of a sudden I can't access precision-for-planning. I guess you and/or Bob are tweaking something. So I can't make sensible comments about much of the above. But I do recall that this version looks good in lots of ways. I agree that the limits on the horiz axis, and step size needs some thought. I have it on the list to ask Bob. He asked for the .005 steps (in target MoE, and rho in the paired design case). Probably needlessly fine-grained. .01 steps probably ok. Arbitrary values for the assurance curve--aha, that supersedes my last comment on issue #1 [https://github.com/gfmoore/esci-precision/issues/1]. 'ESCI intro' is ESCI for ITNS. Of relevance here: 'ESCI intro chapters 10-16'. One of the files I think I sent. The other is 'ESCI chapters 10-13', which is the earlier version, for UTNS, with more detailed and complex precision stuff. (Three sheets, not two; H axis goes to 2.0; includes a z curve.) We mainly want the simpler versions of the two precision sheets in ESCI intro. Agreed that N=1 makes no sense. In both ESCIs I set N = 3 as the min, whether two groups each of N=3, or N=3 in paired design. I'm tempted to set N=3 again, tho' I guess that we could allow N=2, but make that a minimum. On the list to check with Bob. I don't know what chi sq function you use. I did discover that in ESCI (both versions) I used the Excel function CHIINV(prob, df) which is now regarded as superseded, tho' still supported. The modern equivalent is CHISQ.INV(1-prob,df), just to be maximally confusing. So we want, for 99% assurance, e.g. CHIINV(.01,df) == CHISQ.INV(.99,df), to give the critical value of chi-sq with only 1% of the area to the right of this crit value. — You are receiving this because you commented. Reply to this email directly, view it on GitHub [https://github.com/gfmoore/esci-precision/issues/2#issuecomment-714216423], or unsubscribe [https://github.com/notifications/unsubscribe-auth/AFNW2RZ5UZWK2ZRVFSPFU2LSL6YJTANCNFSM4SZHZLWQ].

[gfmoore]

Is there some other colour you could easily use for 'Truncate display' in both tabs? Maybe a buff or light brown you have used elsewhere?

done

target MoE = 0.40 is the suggestion, not the current 0.25.

done

I don't know what chi sq function you use. I did discover that in ESCI (both versions) I used the Excel function CHIINV(prob, df) which is now regarded as superseded, tho' still supported. The modern equivalent is CHISQ.INV(1-prob,df), just to be maximally confusing.

So we want, for 99% assurance, e.g. CHIINV(.01,df) == CHISQ.INV(.99,df), to give the critical value of chi-sq with only 1% of the area to the right of this crit value.

Oops, missed all this, sorry. There's so much info flying around and I get us out of sequence. This days delay is difficult. Imagine if you lived on Mars, we'd get nothing done :)

[gfmoore]

I've done the best I can so far with the assurance calcs. They are okay at the low f, but not quite as higher at the higher f. I'll send a separate email to you and Bob as I think we need to think through the calculations carefully and my math is not up to the job. All I know is that I am concerned that the rounding and forcing values and choosing different values is a bit kludgy. I've tried to keep the code clean, but follow the basic ideas.

I'm closing this as I think I've covered it all and we can open separate issues (even if was covered here).