KM Mean

[BvilleD]

Hi. I was calculating a KM mean and standard deviation using enparCensored and then compared the results to ProUCL 5.2 and got different results. I also tried reproducing the KM mean using kmms in STAND and got consistent results with ProUCL 5.2. I see enparCensored was recently updated, so I'm not sure if there is some new argument that I have failed to set properly or not. The example data are attached. Thanks km.example.data.xlsx

[cfholbert]

@BvilleD The new argument is "restricted." By fault, this is set to FALSE. You need to set it to TRUE (i.e., restricted = TRUE) to get the mean and sd that is generated by ProUCL 5.2. Alternatively, you can install EnvStats 2.8.1, which does not use the restricted argument. However, the previous version does not accurately estimate the SE.

[BvilleD]

Thanks Charles!I apologize for the very delayed response, for some reason this was sent to my junk folder. Setting the restricted argument to TRUE fixed my confusion. What threw me off was the description in the help file that indicated “To be consistent with ProUCL 5.2.0, by default the function enparCensored does not compute the restricted mean (i.e., restricted = FALSE).” So I was thinking the restricted argument should have been FALSE.One more question, I see that when restricted is set to TRUE, a confidence interval is not generated. Is there a reason that estimate is not applicable using a restricted mean?Thanks again for all the help, I really appreciate it!DaveSent from my iPhoneOn Sep 18, 2024, at 12:45 PM, Charles Holbert @.***> wrote: The new argument is "restricted." By fault, this is set to FALSE. You need to set it to TRUE (i.e., restricted = TRUE) to get the mean and sd that is generated by ProUCL 5.2.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

[cfholbert]

@BvilleD @SteveMillard From my testing of EnvStats version 3, the Restricted argument must be set to TRUE to match the mean and sd that is generated by ProUCL. I'm not sure why the doumentation states that Restricted should be set to FALSE to be consistent with ProUCL. I had problems with version 3 replicate results from ProUCL when the smallest value for left-censored data is censored and less than or equal to the smallest uncensored value. Thus, I rolled back to version 2.8.1. Because of this, I can't tell why you are having problems getting confidence limits when using version 3 and Retricted set to TRUE. Perhaps Steve could weigh in on this.

[SteveMillard]

Hi Charles!Thanks for your email. I will be happy to help out. Can you please send me the data set that you were using, and also the are commands that you were using.Just to make sure, are you able to reproduce the results in the Example section of the help file for enparCensored?Sincerely,—Steve M.On Oct 1, 2024, at 7:07 AM, Charles Holbert @.***> wrote: @BvilleD @SteveMillard From my testing of EnvStats version 3, the Restricted argument must be set to TRUE to match the mean and sd that is generated by ProUCL. I'm not sure why the doumentation states that Restricted should be set to FALSE to be consistent with ProUCL. I had problems with version 3 replicate results from ProUCL when the smallest value for left-censored data is censored and less than or equal to the smallest uncensored value. Thus, I rolled back to version 2.8.1. Because of this, I can't tell why you are having problems getting confidence limits when using version 3 and Retricted set to TRUE. Perhaps Steve could weigh in on this.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[SteveMillard]

R commands, not are commands.  That’s what happens when you use dictation :)Also, I am talking about the Example section in the help file for enparCensored for Envstats version 3.0.Sincerely,—Steve M.On Oct 1, 2024, at 12:35 PM, Steve Millard @.> wrote:Hi Charles!Thanks for your email. I will be happy to help out. Can you please send me the data set that you were using, and also the are commands that you were using.Just to make sure, are you able to reproduce the results in the Example section of the help file for enparCensored?Sincerely,—Steve M.On Oct 1, 2024, at 7:07 AM, Charles Holbert @.> wrote: @BvilleD @SteveMillard From my testing of EnvStats version 3, the Restricted argument must be set to TRUE to match the mean and sd that is generated by ProUCL. I'm not sure why the doumentation states that Restricted should be set to FALSE to be consistent with ProUCL. I had problems with version 3 replicate results from ProUCL when the smallest value for left-censored data is censored and less than or equal to the smallest uncensored value. Thus, I rolled back to version 2.8.1. Because of this, I can't tell why you are having problems getting confidence limits when using version 3 and Retricted set to TRUE. Perhaps Steve could weigh in on this.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[cfholbert]

@SteveMillard attached are the data and the code I used. I compared the ProUCL 5.2 mean and sd to those calculated using EnvStats 2.8.1 and 3.0.0. I will see if I can rplicate the examples in the enparCensored help file. R-code.txt

[cfholbert]

@SteveMillard I ran the examples in the enparCensored help file and got the same results as those listed. The unrestricted mean did not match that from ProUCL but the restricted mean did match ProUCL. It was the opposite for the SE.

[SteveMillard]

Hi Charles,

Thank you so much. I am out of the office this week, so I will investigate this issue next week and let you know what I find. Thanks for using EnvStats!!

Sincerely, --Steve M.

On 10/01/2024 3:42 PM PDT Charles Holbert @.***> wrote:

@SteveMillard https://github.com/SteveMillard attached are the data and the code I used. I compared the ProUCL 5.2 mean and sd to those calculated using EnvStats 2.8.1 and 3.0.0. I will see if I can rplicate the examples in the enparCensored help file. R-code.txt https://github.com/user-attachments/files/17217336/R-code.txt

— Reply to this email directly, view it on GitHub https://github.com/alexkowa/EnvStats/issues/43#issuecomment-2387211905, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE3SF4QRJHJIRJ5RT42Z5GDZZMQODAVCNFSM6AAAAABNK3A7I2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBXGIYTCOJQGU. You are receiving this because you were mentioned.Message ID: @.***>

[cfholbert]

Thank you, Steve. Your EnvStats package is one the best R packages and I use it daily in my work.

[SteveMillard]

Hi @cfholbert and @BvilleD, Thank you for your kind words and for asking about the enparCensored() function. Attached is an R script showing how the output from enparCensored() compares with ProUCL for 3 different datasets. In all cases, the results of the two programs are similar, but I cannot tell you why they do not match exactly, and it is not clear to me at this point whether ProUCL computes the restricted mean and associated quantities or the unrestricted mean. I know the organization that took over the maintenance and updating of ProUCL recently, and they are stellar people (Neptune & Company). I have corresponded with them and included the same 3 files I include here (note that I had to change the extension of Test.enparCensored.R to Test.enparCensored.txt because GitHub does not support attaching a file with the extention .R). Neptune & Company will look into this issue and I hope they can figure out why there are differences, as they have the ProUCL source code. Thank you again for using EnvStats!! km.example.data - SPM.txt km.example.data - SPM.xlsx Test enparCensored.txt

[cfholbert]

Hi @SteveMillard, Thank you for taking the time to investigate these discrepancies. I am always impressed by your willingness to help and the thoroughness of your documentation, especially the EnvStats book and the help files.