JOSS Review: Documentation - A statement of need & Example Usage

[njtierney]

RE:

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is? Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).

I think the authors could address this by adding a couple of sentences that answer these questions (taken from the README section of the R packages book):

1. Why should I use it?
2. How do I use it?
3. How do I get it?

This section then goes on to provide a structure for the README:

A paragraph that describes the high-level purpose of the package.

An example that shows how to use the package to solve a simple problem.

Installation instructions, giving code that can be copied and pasted into R.

An overview that describes the main components of the package. For more complex packages, this will point to vignettes for more details. This is also a good place to describe how your package fits into the ecosystem of its target domain.

Based on the current paragraph of information in the README, they state:

R package provides a good platform for designing and planning trials with multiple treatment arms. The package provides functionality for designing trials with continuous, ordinal and survival outcomes which offers great flexibility and makes it a comprehensive toolkit to handle different kinds of trials.

To me this indicates that this is to do with medical trials? Or clinical trials, perhaps? I am not sure what "multiple treatment arms" means - it makes me wonder what "single treatment arms" are? Based on what they have said here, I would also expect to see some example workflows of how this package works with designing trials, and the sorts of questions that come up when designing trials, and how this fits in to the puzzle.

The authors provide an example of usage:

#For designing a trial with continuous outcome.
design_cont(delta0 = 0.178, delta1 = 0.545, alpha = 0.05, beta = 0.1, K = 3, frac = c(0.5, 1))

#Generating operating characteristics of the trial.
op_power_cont(alpha = 0.05, beta = 0.1, K = 3, frac = c(0.5, 1), delta0 = 0.178, delta1 = 0.545, nsim = 10000, seed = 10)

But I think that they should first describe what kind of data continuous or operating characteristics means? Perhaps with an example to an existing study, or some existing example data.

The authors then state:

For detailed usage of the package, please see the paper in the repository.

I think the paper.md file should be converted into a vignette that gives more detail about the usage of the software.

I think that by writing more substantial documentation this would also help cover the "example usage":Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).

Link to review: https://github.com/openjournals/joss-reviews/issues/6322

[Tpatni719]

Thank you for the comments! I have explained the example mentioned in the Readme section and the operating characteristics is a common jargon in clinical trial settings so I don't think it is necessary to expound it. Regarding adding the vignette, I think the paper will be a good addition as a vignette at CRAN and now, the results of all the examples along with their comprehensive explanation is also included in the paper.

[njtierney]

Great!

I still think that this first section needs to be unpacked with more detail:

ggMAMS R package provides a good platform for designing and planning trials with multiple treatment arms. The package provides functionality for designing trials with continuous, ordinal and survival outcomes which offers great flexibility and makes it a comprehensive toolkit to handle different kinds of trials.

Specifically, here is a small breakdown of what I mean.

ggMAMS R package provides a good platform for designing and planning trials with multiple treatment arms.

• What does "good platform" mean?
• Designing and planning what kind of trials?
• What are multiple treatment arms? What are single treatment arms?
• What specific problem does this solve within these?
• How does it help with design and planning?
• What information does it help provide the user of the package?

The package provides functionality for designing trials with continuous, ordinal and survival outcomes which offers great flexibility and makes it a comprehensive toolkit to handle different kinds of trials.

• Can you tell me more about why there are certain outcomes?
• Can you give me an example of the different kinds of trials? Medical trials? Drug trials? Just a couple of examples would be useful.

I have explained the example mentioned in the Readme section and the operating characteristics is a common jargon in clinical trial settings so I don't think it is necessary to expound it.

Could you use a README.Rmd file so we can see the output of the example above? usethis::use_readme_rmd() will help generate the file.

The operating characteristics of the trial can be generated using the op_power_cont() functions for power under alternative hypothesis.

Can you describe how these are used, and what they do?

[Tpatni123]

Sure, I can add some details for the mentioned points in the Readme section(most of them are already mentioned in the paper) but I don't understand what you meant by "why there are certain outcomes".

[njtierney]

It's great that you've mentioned them in the paper! But most users will read the README first, so I think it's important to make it as easy as possible for users of a variety of backgrounds to go from the README to understand how to use the package.

"why there are certain outcomes".

Sorry I should have been clearer - some examples of the the type of data or trial that contains continuous, survival, or ordinal outcomes.

[Tpatni719]

The details have been added.

[njtierney]

Thanks!

Just to confirm, it looks like you are manually updating the README.md with R code output - are you using a README.Rmd file or?

[Tpatni123]

Yes, I manually updated it.

[njtierney]

That is a lot of copying and pasting! You might really benefit from using Rmarkdown files, have you seen then before?

If you use:

# if not already installed
# install.packages("usethis")
usethis::use_readme_rmd()

This will generate a special file that you can provide the code in, then click the "knit" button in Rstudio, it will then mean you won't need to carefully copy and paste and format the code.

[Tpatni123]

Thank you for sharing this but it's just that the results of operating characteristics for 10,000 simulations take time and I already had the results so that's why I just copied the results.

[njtierney]

Sure I can appreciate that you don't want it to take too long. However you've already done the hard work and made it so users can control the number of simulations.

I've written a pull request which implements this here: https://github.com/Tpatni719/gsMAMS/pull/15

In doing so I also found a few bugs from the lowercase conversion, and took the liberty to fix those, I hope you don't mind! Those changes are found here: https://github.com/Tpatni719/gsMAMS/pull/14

So you'll need to merge https://github.com/Tpatni719/gsMAMS/pull/14 first, then https://github.com/Tpatni719/gsMAMS/pull/15.

Another bug that was caught was that in op_power_cont the argument K was changed to p.

[Tpatni123]

Thank you so much for your help! The reason you found those weird syntax because some code was kind of complicated down the pipeline so just to be cautious, I did that untidy trick so that all the code remains intact and the code won't break down after changing the argument to lower k. So, that's why I just used K=k to make sure that the code doesn't break down. Regarding the k changed to p in operating characteristics functions, it was intentional as I used k for some other operation inside the function. So, please let me know your thoughts.

PS: I have merged the branches and I will check the functions. Thanks for the help again!

[njtierney]

The reason you found those weird syntax because some code was kind of complicated down the pipeline so just to be cautious, I did that untidy trick so that all the code remains intact and the code won't break down after changing the argument to lower k. So, that's why I just used K=k to make sure that the code doesn't break down. Regarding the k changed to p in operating characteristics functions, it was intentional as I used k for some other operation inside the function. So, please let me know your thoughts.

Yes I could see this - it was one way to solve the problem, however in doing so it introduced some fragility into the package. To help resolve this I did a find and replace - which you can do with Cmd/Ctrl+F and searched for the new old arguments, being sure to update them. I hope that makes sense.

Now that there are tests, and that the README is an Rmarkdown document, you should be able to catch these errors.

[Tpatni123]

Thanks for the help again! And please let me know if I can close this issue.

[Tpatni123]

Hi @njtierney, did you get the chance to review the responses?

[njtierney]

I am happy with this, thanks!