Abstract, Introduction, Conclusion, and Project Management

Tasks:

Abstract: Write a succinct and informative abstract that provides a brief summary of our project.
Introduction: Develop an introduction that gives background on the project and outlines the scope and purpose of our research.
Conclusion: Summarize the key findings and implications of our project, tying everything together in a meaningful way.

Responsibilities:

Organizing the Team: Ensure that everyone on the team understands their roles and responsibilities and has the resources they need to fulfill their tasks effectively.
Managing Progress: Regularly check-in with the team members to see their progress, help them overcome any challenges, and ensure that the tasks are moving forward as planned.
Checking Deadlines: Keep track of the deadlines for each task and ensure that the work is being completed on time.
Assisting Team Members: Be available to assist team members with their tasks as needed, provide guidance and feedback, and foster a supportive and collaborative team environment.

Assignees: Xinyi, Hongzhang

Plan for Abstract, Introduction, Conclusion, and Project Management mini-group:

1. Introduction: We know that this part and the interview part may generate duplication. Therefore, before we have the draft of the literature review (before 5/16), we will focus on the following: a) Why do we need to design a dose-finding study? b) Why do we need to use continual reassessment methods (CRM)? Why is CRM better than the traditional 3+3 design? After we have the results of the literature reviews and simulation studies, we will then try to solve the following: c) What are the relationships between our simulation results and the CRM? In other words, why is our finding (the difference in the performance and the reason for the difference) important in the CRM methods? We plan to finish the introduction part on 5/20. And continue updating it until the final project presentation.

2. Abstract and Conclusion: In this part, we plan: a) Learning and understanding all the results from the literature review and simulation groups. b) Making sure our results can answer all the tasks in the final project requirements. And make sure the logical chain between each part is rigorous enough. c) Trying to extract our main findings and main conclusions and express them in clear words. d) Discussing with all the group members, collecting suggestions, and making revisions

We plan to finish the abstract and conclusion part on 5/20. And continue updating it until the final project presentation.

3. Slides for the presentation a) After we finish the above two parts, we plan to collect and organize the results made by each member and make 10-25 pages of slides for the presentation. b) We plan to finish the slides before 5/23. And at 5/23 or 5/24, we plan to make a mock presentation, collect suggestions and make final adjustments.

4. Management: During the whole final project, we are responsible for the following: a) Organizing the Team: Ensure that everyone on the team understands their roles and responsibilities and has the resources they need to fulfil their tasks effectively. b) Managing Progress: Regularly check in with the team members to see their progress, help them overcome any challenges, and ensure that the tasks are moving forward as planned. c) Checking Deadlines: Keep track of the deadlines for each task and ensure that the work is being completed on time. d) Assisting Team Members: Be available to assist team members with their tasks as needed, provide guidance and feedback, and foster a supportive and collaborative team environment. Xinyi is the main person for the Management, and Hongzhang will also help her.

Thanks for the great plan, Hongzhang! I think it looks good! ✨

Hi team, I finisded our introduction. I will countinue to modify this until the presentation. Is there some parts needs modify? I'm very expect to receive your suggestions!

Introduction

Phase I dose-finding clinical trials are conducted to analyze the safety and tolerability of a new drug. This is a critical step in drug development as it marks the first time the new drug is used in humans. The trials aim to determine the Maximum Tolerated Dose (MTD) and give a foundation for further drug development and clinical trials.

Because we are not fully understood the actions and side effects of new drugs in the human body, there is a potential risk to the trial participants. Also, experimenting with new drugs at low doses level may delay patients' chances of treatment. On the other hand, Phase I trials typically require close health monitoring of participants, which requires substantial medical resources and specialized personnel. Therefore, considering ethical and cost factors, phase I dose-finding trials often have a small sample size, with each dose-level cohort comprising only around three participants.

Why frequency not suitable for phase I dose-finding trials?

The traditional frequency method presumes a fixed coefficient for the Dose Limiting Toxicity (DLT) distribution under the same dose level. Each sample point is an independent draw. Given a large enough sample size, the sampling mean will finally equal the actual coefficient. Hence, we can make a hypothesis test for finding the MTD (such as the assumption that DLT at a particular dose level smaller than 0.33). If we observe enough extreme data based on the null hypothesis, we can reject this hypothesis. However, as the frequency hypothesis test is based on the law of large numbers, there will be a huge variance in phase I dose-finding trials because of the small sample size. In such cases, the likelihood of a type II error (accepting an incorrect null hypothesis) would be very high. The results of the hypothesis test are not robust.

On the other hand, in the process of drug research and development, we often have some prior information about the drug (such as the results of animal experiments). This information is of great help in determining the MTD. However, the frequency method cannot make good use of this information.

Finally, the conclusion of the hypothesis test only rejected or failed to reject. Even if we reject the null hypothesis (e.g. H0: DLT < TTL), the hypothesis test still says nothing to the H1. In other words, rejecting the null hypothesis (H0) just means that the probability of observing the current data is extremely small when H0 is true. It does not give us any evidence for H1 (e.g. DLT >= TTL). We do not know the degree of confidence for DLT >= TTL. But in phase I dose-finding trials, clinicians need this information to decide the following trial strategy. Therefore, it is not suitable for the frequency method in phase I dose-finding trials.

Traditional 3+3 method

Currently, traditional 3+3 method is the mainstream method for determining MTD in phase I dose-finding trials. By the literature review from Tongtong Jin, more than 90% of phase I dose-finding trials use rule-based designs such as traditional 3+3.

The 3+3 method is intuitive, easy to operate, and provides a clear and repeatable method to confirm the MTD, which can be more easily understood and accepted by clinical researchers, approval agencies, and ethics committees with non-statistical backgrounds. At the same time, the 3+3 method is cautious about increasing the dose level and has been tested by many previous studies, which shows that 3+3 has good safety.

However, a large number of studies have pointed out that the 3+3 method also has some disadvantages.

a. The 3+3 method cannot efficiently utilize all the information in the trials. The decision at each dose level depends only on the information of the three (or six) experimental participants at the current dose level. The 3+3 method does not consider the information outside this dose level.

b. In the process of increasing the dose level, at least three subjects are required for each dose level, and the clinicians should wait for the result in the current dose level to make the decision to increase the dose level. Even if the current results show that the preset measurement increases too slowly, it is hard to adjust the dose level during the experiment according to the new information. This may expose more patients to ineffective low doses level treatment and may also delay the overall trial period. (This also shows that the 3+3 method lacks flexibility).

c. When determining MTD, the percentage of DLT is fixed at 33%. The TTR (Target toxicity rate) cannot be set flexibly.

d. Unable to establish a dose-response curve: The 3+3 method can only find a fixed MTD, but cannot build a model to show the relationship of the toxicity in different dose rate.

CRM method

After each trial in a certain dose level, CRM will update the model based on all the data (posterior) and then use this updated model to predict the toxicity risk of each dose level. Based on these results, the clinicians will decide on the next dose level and continue the trial until meeting the stopping rules. Compared with the traditional 3+3 method, CRM has the following advantages:

a. Compared with 3+3 methods, CRM makes better use of all the data that has been collected during the phase I dose-finding trials. In other words, CRM is not only considered the information in the current dose level (3+3) but also considers the DLT information of all previous doses. So CRM can estimate MTD more accurately.

b. In the process of phase I dose-finding trials, CRM can easily adjust the subsequent test design and dose level according to the existing experimental information. Therefore, CRM can reduce the sample size and time wasted during the low-dose level (or reduce the risk of severe toxic reactions by reducing the dose level in the next trial), thereby reducing the risk of patients' exposure to extremely high or low dose levels (high DLT risk or the risk of delayed treatment).

c. CRM can analyze the complex relationship between dose and toxicity by constructing different Dose-toxicity skeleton models and can adapt to different situations by changing the model's parameters. For example, suppose there are two drugs, the DLT probability of the first drug is linearly related to the dose level. The DLT probability of the second drug is extremely low at low dose levels, and the DLT probability increases rapidly after a certain dose level. In the 3+3 method, the strategies of the two drugs are exactly the same, while in the CRM, we can use the power model for the first drug and the logistic model for the second drug for a better fit of the model.

d. The CRM method uses the Bayesian method to construct a dose-response curve, which can better reflect the relationship between toxicity and dose, and can be used to estimate the MTD at any TTR level.

Why our final project important?

Although the continuous reassessment approach (CRM) has many advantages, it also has some potential disadvantages and limitations:

a. CRM is a relatively complicated method, and there is no mature SOP (standard operating procedure). This means that the experiment needs a statistical expert for inference and decision-making, and the model may be too complex for the clinical team and the SMC (Safety Monitoring Committee or Data and Safety Monitoring Board) member. This has become one of the main barriers to the uptake of CRM methods in phase I dose-finding trials.

b. The dose level in the CRM method increases rapidly under some models (especially single-parameter models), which may increase the risk of the number and severity of DLT while improving the efficiency of the experiment. For this reason, some ad-hoc rules may be needed to control the risk of overdose.

c. The implementation of CRM requires the clinical team to choose appropriate models and parameters. Improper selection or setting may affect the results of CRM. Especially in the early stages of the trial, due to the lack of data, the results of CRM may have large fluctuations in the different prior models.

It should be noted that the development of a drug often has a large amount of research investment and potential marketing profits. Due to the high flexibility of CRM, the clinical team may modify the model base on their personal profits but not based on statistics and medical theory, even though this may put patients at a higher risk of DLT. Therefore, we need to further analyze the effect of models and parameters setting on CRM inference and then establish a complete SOP and avoid the problem of that events.

In our final project, we first conducted a literature review on Wheeler et al. l (2019) and twenty relevant studies. Subsequently, we use the R-studio to make simulations to find the difference between two scenarios with different true toxicity probabilities ((0.25, 0.3, 0.5, 0.6, 0.7) vs (0.01, 0.05, 0.2, 0.3, 0.5)) on the estimated MTD in the CRM model. Finally, we discussed why this difference happens and how to avoid this problem.

We believe that our final project research can help us better understand the impact of toxicity probabilities difference on the MTD results in the CRM model. This finding can help the clinical team estimate MTD more accurately and helps in making the mature SOP of CRM in phase I dose-finding trials.

Attaching the current version of this part thanks to the impressive work of @HongzhangXie.🌟

bayes_final_project_intro_v1.pdf

Conclusion

In the literature review part, we find rule-based designs like the 3+3 are still more prevailingly used in the phase 1 trials now, but many studies show that model-based designs like CRM are performing better than rule-based designs. The CRM model can use all the toxicity information during the trial, reduce excessive exposure to subtherapeutic doses, reach the MTD faster (fewer patients) and make more accurate MTD. We also find there is a fast development of CRM both in theory and in the industry.

In the simulation part, we find the starting point of the dose level is important for the performance of the CRM. In the two different Scenarios, the difference in start point may cause around a 10%-20% difference in the probability of finding the true MTD (47% vs 69%). And in the classic CRM model, a conservative starting point (in other words, start at a relatively low dose level) have better accuracy in estimating the true MTD. Therefore, we recommend that a relatively conservative dose level strategy be adopted in clinical trials SOP. This can increase both the accuracy of CRM and the safety of the experiment.

Finally, we tried some methods to mitigate the difference in CRM performance in estimating MTD. We tried several methods like make penalization for underdosing and adding extra lower dose levels if the distribution is too skewed to the right. We find the strategy both “force the selection to go to dose 3 instead of dose 6 if first three patients do not get sick” and “limit the analysis to simulations where the sampling does not just stick to one dose (< 90% cutoff)” can efficiently solve this problem. The accuracy for the high start point scenario increases from 48% to 62%.

I try to write a new version of the conclusion

Conclusion

Finding the maximum tolerated dose (MTD) of a new therapy in Phase 1 dose-finding clinical trials is an essential step in drug development. We can use rule-based approaches, such as the 3 + 3 design or model-based designs, such as the continual reassessment method (CRM) in the phase I trials. In the introduction part and literature review part, we discussed the advantages and disadvantages of those two designs. We find rule-based designs are still more prevailingly used in phase 1 trials due to lits simplicity and ease of use. But a large number of studies have shown that the CRM method has a better performance than the traditional 3 + 3 method because it is more flexible and can use all the information. Due to its advantages, CRM has attracted the attention of many researchers. This method has been continuously developed in subsequent studies to improve the accuracy of finding MTD, make it easier to use in clinical and reduce the potential risk during the phase 1 trials.

In the methodology and model reproduction part, we discussed how to use the Bayesian and likelihood methods in the CRM and why we use the logistic regression model in our project. Then we use the R-studio to reproduce the results in Figures 6 & 7.

In the simulation part, our aim is to analyze the performance of the CRM method in two different scenarios. Controlling other conditions unchanged, how the different true toxicity probabilities ((0.25, 0.3, 0.5, 0.6, 0.7) vs (0.01, 0.05, 0.2, 0.3, 0.5)) will impact the accuracy of CRM to find the true MTD. With the target toxicity rate of 0.33, simple size 36 and cohort size 3, we make 1000 times simulation each case and find there is around a 21% difference in the probability of finding the true MTD (48% vs 69%). We attribute this difference to two main reasons. Firstly, in both scenarios, there is a reasonable chance that all three patients will not get sick (42% and 99%) and jump to the highest dose level (dose six). For scenario 1, dose 6 is much further from the true dose and hard to climb down to the real MTD level. So the wrong doses are oversampled more often in scenario 1. The other reason for the difference is the gap between the MTD (0.3) and the next closest dose (0.25) in scenario 1 is only 5%, which is more closely related to scenario 2 (10%). This also increases the probability of selecting the wrong MTD in scenario 1.

To mitigate the difference, we try to propose three strategies:

a. Use Bayes risk and Bayesian decision theory for posterior probability intervals associated with each dose as the inferential summary. In this strategy, we use the dose level with the lowest Bayes risk as the dose level for the next cohort. However, we did not find a significant improvement in the accuracy of this strategy.

b. Toss-out samples plus early alterations. In this strategy, we both “force the selection to go to dose 3 instead of dose 6 if first three patients do not get sick” and “limit the analysis to simulations where the sampling does not just stick to one dose (< 90% cutoff)”. After the modification, the accuracy in scenario 1 increased from 48% to 60%. And there is only a little difference in the accuracy in scenario 2 (64% to 62%). We think this strategy can effectively mitigate the difference caused by the start point.

c. Conditionally adding extra lower dose level. We observe the results at the 1/3 cut point of the trail and evaluate the dose visit frequency. If the distribution is too skewed to the right, we will add an extra lower level 0 for the remaining 2/3 patients. We can also find a significant improvement in the performance in scenario 1. But it may be difficult to implement in real-world clinical trials.

xinyi030 / PHS43010_NonStatGroup

Abstract, Introduction, Conclusion, and Project Management #2