CRAN task view proposal: ClinicalTrials

ya-wang-git commented 1 year ago

Prior to this proposal we have reached out to the CRAN task view editors regarding a possible update of the existing ClinicalTrial task view. They advised using this issue to propose a relaunch that can then be discussed.

Scope

This task view aims to collect information on R packages for clinical trial design, monitoring, analysis and reporting.

Packages

Our plan is to start out from the existing task view (link) and use our pre-specified workflow to determine whether a package should be kept or removed from the task view. The workflow could comprise different steps, e.g. first search according to a broad keywords list, and second manual review.
In addition, we would merge lists of relevant clinical trial R packages being used by openstatsware members and their organizations and include those in the task view.
The existing task view groups R packages into the following categories: (1) design and monitoring, (2) design and analysis, (3) analysis for specific designs, (4) analysis in general, (5) meta-analysis. We propose to have more hierarchies such that packages of the same theme can be grouped together and become easier to explore. For example, within study design, packages for dose-finding purpose can be grouped together as a subsection, and those for the sample size calculation could be another subsection.
In addition, we propose to have a “Getting Started” section at the beginning of the task view to point users to relevant packages or introduction material.
In a future version, we would consider adding further metadata information, e.g. in the form or labels or links, regarding the quality of the listed R packages. We note that this could also be done on a more general level across CRAN task views later.

Overlap

Potential overlap with the task view ExperimentalDesign whose focus is on R packages for experimental design and analysis of experimental data: Our proposed task view focuses more on R packages that are designed for clinical trials.
Potential overlap with the task view Survival which provides a comprehensive list of R packages for the analysis of time to event data: Again, our proposed task view focuses only on R packages that are designed for clinical trials.
Potential overlap with the task view Meta-analysis which covers R packages for meta-analysis of summary statistics from primary studies. Similarly, our proposed task view focuses only on R packages that are designed for clinical trials.
For R packages that are designed for the analysis of pharmacokinetic data, we would refer users to the existing task view Pharmacokinetics by providing cross reference (in the same way as currently the Pharmacokinetics task view refers back to Clinical Trials task view).

Maintainers

The maintainers are all from the openstatsware working group.
Principal maintainers: Ya Wang, Elias Laurin Meyer
Co-maintainers: Laura Pascasio Harris, Orla Doyle, Wilmar Igl

zeileis commented 1 year ago

Ya @ywang-gilead et al., thank you for the initiative and, as previously discussed via e-mail, I think that a relaunch of the task view would be good to make it more active and dynamic again. Also, I think it's great to build on existing initiatives under the umbrella of the ASA and R Consortium. However, some more work is needed before we can move forward:

The package inclusions/exclusion guidelines are a too vague. You say that you want "more hierarchies" but it is unclear to me what is the intended structure of the task view. And how will this help to determine whether a package should be included in the task view or not.
Similarly, more concrete discussion of how to handle the overlap with other task views is needed. For example, the ExperimentalDesign task view has an explicit list of packages for design of clinical trials and they suggest that this should be moved to the ClinicalTrials task view. Do you concur with this list or would you resolve this differently? Explain your strategy. My impression is that the issues are similar for Pharmacokinetics where there could be more overlap while Survival and MetaAnalysis are separated a bit more clearly.
You indicate that you want to move in the direction of labeling the quality of R packages, both within this task view and even across task views. Note that this is not within the scope CRAN Task Views. As the Documentation says: "The views are intended to have a sharp focus so that it is sufficiently clear which packages should be included (or excluded) - and they are not meant to endorse the "best" packages for a given task". The Proposal guidelines explain this in a bit more detail:

Ratings: Task views should not rate the packages or endorse certain "best" packages but rather give an overview of what is available. A bit of emphasis to the more important packages can be given in two ways: (1) The most important packages can be flagged as "core" packages. (2) In the information text the more important packages can be listed first in the respective sections.
Regarding the maintainers: First, there has to be a single principal maintainer who is the principal contact for the task view. Thus, it's not possible to have two principal maintainers. Second, having several people from the openstatsware working group is fine because this is already a rather diverse intiative across companies. But rather than having all five contributors from the openstatsware group, I think it would be good to have some outside co-maintainers as well, ideally also including someone from academia.

You don't have to respond to my comments right away but we can also wait for some more comments from my fellow CRAN Task View Editors @rsbivand @eddelbuettel @tuxette

Also I'm tagging here the maintainers of the other task views mentioned in case they want to add to the discussion: @ugroempi @tylermorganwall @aallignol @deweyme @wviechtb @billdenney

billdenney commented 1 year ago

I don't know the specific quality metrics being considered. But, I have seen the quality metrics mentioned in multiple ways for R packages used in pharmaceutical development. The ways that I've seen them used align with some R core principles, and they are not necessarily ranking or endorsement but they are exposing objective quality metrics such as test coverage percent, maintenance activity, and duration of time on CRAN.

As it relates to the Pharmacokinetics view, I would assume that the Clinical Trials view would reference the Pharmacokinetics view. And, I would assume that there would be little overlap between packages.

deweyme commented 1 year ago

Dear All

Most of the packages in the MetaAnalysis CTV could, in principle, be used for meta-analysis of clinical trials but none of them is restricted to such designs. So I would suggest a cross-reference from each to the other would suffice.

As far as quality is concerned I have not included a couple of packages in the past. One did not really do what it said on the tin and at least one more I completely failed to understand what it was claiming to do even after reading the entire manual. Apart from that they all go in and I leave it up to the user to check against their own criteria.

Michael

On 26/10/2023 02:51, Achim Zeileis wrote:

Ya @ywang-gilead https://github.com/ywang-gilead et al., thank you for the initiative and, as previously discussed via e-mail, I think that a relaunch of the task view would be good to make it more active and dynamic again. Also, I think it's great to build on existing initiatives under the umbrella of the ASA and R Consortium. However, some more work is needed before we can move forward:
The package inclusions/exclusion guidelines are a too vague. You say that you want "more hierarchies" but it is unclear to me what is the intended structure of the task view. And how will this help to determine whether a package should be included in the task view or not.

Similarly, more concrete discussion of how to handle the overlap with other task views is needed. For example, the |ExperimentalDesign| task view has an explicit list of packages for design of clinical trials and they suggest that this should be moved to the |ClinicalTrials| task view. Do you concur with this list or would you resolve this differently? Explain your strategy. My impression is that the issues are similar for |Pharmacokinetics| where there could be more overlap while |Survival| and |MetaAnalysis| are separated a bit more clearly.
You indicate that you want to move in the direction of labeling the quality of R packages, both within this task view and even across task views. Note that this is not within the scope CRAN Task Views. As the Documentation https://github.com/cran-task-views/ctv/blob/main/Documentation.md says: /"The views are intended to have a sharp focus so that it is sufficiently clear which packages should be included (or excluded) - and they are not meant to endorse the "best" packages for a given task"./ The Proposal guidelines https://github.com/cran-task-views/ctv/blob/main/Proposal.md explain this in a bit more detail:
Ratings: Task views should not rate the packages or endorse
certain "best" packages but rather give an overview of what is
available. A bit of emphasis to the more important packages can
be given in two ways: (1) The most important packages can be
flagged as "core" packages. (2) In the information text the more
important packages can be listed first in the respective sections.
Regarding the maintainers: First, there has to be a single principal maintainer who is the principal contact for the task view. Thus, it's not possible to have two principal maintainers. Second, having several people from the openstatsware working group is fine because this is already a rather diverse intiative across companies. But rather than having all five contributors from the openstatsware group, I think it would be good to have some outside co-maintainers as well, ideally also including someone from academia.
You don't have to respond to my comments right away but we can also wait for some more comments from my fellow CRAN Task View Editors @rsbivand https://github.com/rsbivand @eddelbuettel https://github.com/eddelbuettel @tuxette https://github.com/tuxette

Also I'm tagging here the maintainers of the other task views mentioned in case they want to add to the discussion: @ugroempi https://github.com/ugroempi @tylermorganwall https://github.com/tylermorganwall @aallignol https://github.com/aallignol @deweyme https://github.com/deweyme @wviechtb https://github.com/wviechtb @billdenney https://github.com/billdenney

— Reply to this email directly, view it on GitHub https://github.com/cran-task-views/ctv/issues/59#issuecomment-1780291870, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW7BJ5HIJZE2QXFCWQAKUH3YBG62JAVCNFSM6AAAAAA6QHDST6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBQGI4TCOBXGA. You are receiving this because you were mentioned.Message ID: @.***>

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient Virus-free.www.avg.com http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

-- Michael

tuxette commented 1 year ago

Thank you for the proposal. I have not much to add to @zeileis 's comments. There might be a minor overlap with Epidemiology as well.

ya-wang-git commented 10 months ago

Thanks a lot for the comments. We have incorporated them into our revised proposal as follows.

Scope

This task view aims to collect information on R packages for clinical trial design, monitoring, analysis and reporting.

Packages

Our plan is to start out from the existing task view (link). A package will be flagged as the “core” package if it provides a wide range of functions (e.g., clinfun can be used for study design, data analysis and sample size calculation), or it is a classic package and has been on CRAN for a long time (e.g., survival), or it has been the dependency package of many other packages (e.g., survival). A package will be excluded if all maintainers find it difficult to understand.
In addition, we would merge lists of relevant clinical trial R packages being used by openstatsware members and their organizations and include those in the task view.
The existing task view groups R packages into the following categories: (1) design and monitoring, (2) design and analysis, (3) analysis for specific designs, (4) analysis in general, (5) meta-analysis. We propose to have more theme-based hierarchies such that packages of the same theme can be grouped together and become easier to explore. For example, within study design, packages for dose-finding purpose can be grouped together as a subsection, and those for the sample size calculation could be another subsection. Proposed themes are as follows:
- Design
  - Adaptive study design
  - Group sequential design
  - Dose finding
  - Randomization
  - Sample size calculation
  - Simulation
  - …
- Monitoring
- Analysis
  - General analysis
  - Longitudinal data analysis
  - Survival analysis
  - Meta-analysis
  - …
- Reporting
In addition, we propose to have a “Getting Started” section at the beginning of the task view to point users to relevant packages or introduction material.

Overlap

Potential overlap with the task view ExperimentalDesign whose focus is on R packages for experimental design and analysis of experimental data: Our proposed task view focuses more on R packages that are designed for clinical trials. For overlap packages, we propose to discuss and align with its maintainers.
Potential overlap with the task view Survival which provides a comprehensive list of R packages for the analysis of time to event data: Again, our proposed task view focuses only on R packages that are designed for clinical trials. And we propose to give cross reference to the Survival task view.
Potential overlap with the task view Meta-analysis which covers R packages for meta-analysis of summary statistics from primary studies. Similarly, our proposed task view focuses only on R packages that are designed for clinical trials. And we propose to give cross reference to the Meta-analysis task view.
For R packages that are designed for the analysis of pharmacokinetic data, we would refer users to the existing task view Pharmacokinetics by providing cross reference (in the same way as currently the Pharmacokinetics task view refers back to Clinical Trials task view).

Maintainers

We have five maintainers from openstatsware and one from academia.
Principal maintainer: Ya Wang (Gilead Sciences)
Co-maintainers:
- Elias Laurin Meyer (Berry Consultants)
- Laura Pascasio Harris (Denali Therapeutics)
- Orla Doyle (Novartis)
- Wilmar Igl (ICON)
- Thomas Jaki (Professor of Statistics at MRC Biostatistics Unit and University of Regensburg)

wviechtb commented 10 months ago

I have no general objections to the proposal. As co-maintainer of the Meta-analysis task view, I will just reiterate what @deweyme already touched on. It isn't really clear to me how one would define an R package for meta-analysis that is "designed for clinical trials". The same applies to the potential overlap with other task views (like ExperimentalDesign and Survival and 'Longitudinal data analysis' also overlaps with the MixedModels task view), that is, how does one determine whether a package from these other task views is designed for clinical trials?

tuxette commented 10 months ago

@ywang-gilead : Thank you for your answer and the clarification / corrections. I think that they cover most of @zeileis 's comments. Additional remarks:

Your choices for core packages sound relevant but you still have to clarify inclusions/exclusion guidelines (probably by a short paragraph at the beginning of the task view). In relation with @wviechtb 's comment, I agree that it will not always be easy to define if an R package is a better fit for ClinicalTrial or for some other related task view. However, linking the other task view might be a solution (in combination with duplicating some of the most important packages if necessary). For instance, https://cran.r-project.org/web/views/OfficialStatistics.html#imputation has a link, a short discussion, and highlights two particularly relevant packages for both OfficialStatistics and MissingData.
Your proposal for themes should indeed clarify the task view. Depending on the number of packages in each theme, note that they can just be bullet "titles" in a list, like in https://cran.r-project.org/web/views/SpatioTemporal.html (in order to not overload the task view with many subsections).
Your proposal for a “Getting Started” section at the beginning of the task view is OK with me but be careful that the main point of the task view is to provide information on CRAN packages (so do not overload it with material on the topic). Also, as already pointed by @zeileis , this section should not sound like it endorses the "best" packages. For instance, https://cran.r-project.org/web/views/FunctionalData.html has a similar section, which presents the most general packages for the topic.
As far as I can tell, you complied to the initial request on maintainers.

I let @zeileis @rsbivand @eddelbuettel react as well but I think that the proposal is interesting and that you can go on working on it.

mstackhouse commented 9 months ago

Hi @ywang-gilead,

We were directed here through our proposal for a new task view for the pharmaverse collection of packages, which we've submitted here. Per @zeileis, the suggestion is that may be logical to fold our collection of packages into the updates relevant to this proposal.

We've drafted the task view, which you can additionally review here and see why we've suggested a specific scope of packages.

Could you please share your thoughts on if/how you see value in combining these packages within the ClinicalTrials task view?

ya-wang-git commented 9 months ago

Hi @mstackhouse ,

Thanks a lot for the additional information. I'm thinking of scheduling a meeting for us (CTV ClinicalTrials maintainers and pharmaverse maintainers) to discuss the possibility to combine these two task views and how we could collaborate. I will send out an email to everyone and hopefully we can find some time next week that works for all of us. Does that sound good to you?

mstackhouse commented 9 months ago

Just wanted to leave a note here that both groups of maintainers met as a team. We mutually decided that right now it's better to keep our proposed task views separate and reassess in the future if it makes sense to combine them. We will continue to address feedback in #60

wiligl commented 8 months ago

Hi, I am not sure that the current Task View concept as a narrative list of packages for a specific scope is the best solution to inform users of relevant packages and whether this will scale with the increasing number of packages. I would suggest to allow package developers to add a field in the description file which says in which task views their package should be listed. The Task view could then be compiled automatically, eg the package name and description (oneliner) could be automagically extracted from the Description file. The Task view page/table could also show additional quality attributes for each package to guide users (first release, last release, number of users, user rating, author credibility, ...).

deweyme commented 8 months ago

Dear Wilmar

In the MetaAnalysis CTV we try to provide some structure to the list of packages. It is hard to see how the software you propose would know which section(s) the new package might appear in. Several of the packages appear in more than one section. If we abandon the idea of curating and organising the list it will become much harder for people to find the relevant package which might fit their needs.

Michael

On 12/03/2024 15:35, Wilmar Igl wrote:

Hi, I am not sure that the current Task View concept as a narrative list of packages for a specific scope is the best solution to inform users of relevant packages and whether this will scale with the increasing number of packages. I would suggest to allow package developers to add a field in the description file which says in which task views their package should be listed. The Task view could then be compiled automatically, eg the package name and description (oneliner) could be automagically extracted from the Description file. The Task view page/table could also show additional quality attributes for each package to guide users (first release, last release, number of users, user rating, author credibility, ...).

— Reply to this email directly, view it on GitHub https://github.com/cran-task-views/ctv/issues/59#issuecomment-1991938918, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW7BJ5CNZJ4RUTTY6T5ZFSTYX4OFVAVCNFSM6AAAAAA6QHDST6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJRHEZTQOJRHA. You are receiving this because you were mentioned.Message ID: @.***>

http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient Virus-free.www.avg.com http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

-- Michael

zeileis commented 8 months ago

Wilmar, thanks for your feedback. There are indeed different potential solutions for generating topic-related lists of packages - with differents strengths and different drawbacks. As Michael already explained, CRAN Task Views aim to be curated lists with a sharp focus. When package maintainers can self-select their package into the task views, this sharp focus would likely be lost. Both because some less relevant packages would be included - but also because some very relevant packages would not be. To learn more about the ideas and strategies behind the CRAN Task View Initiative, you can have a look at this paper: doi10.48550/arXiv.2305.17573:

tuxette commented 3 months ago

Hi @ywang-gilead and col. We were very close to a final version for this proposal: do you want to implement the last suggested changes (see comments since your last proposal) so that we can proceed to the publication?

ya-wang-git commented 3 months ago

Hi @tuxette, we will implement the last suggested changes and provide a draft task view for review.

cran-task-views / ctv