reviewer-3-comment-5/6

[docsteveharris]

I've grabbed these references

1. Davis SE, Lasko TA, Chen G, Siew ED, Matheny ME. Calibration drift in regression and machine learning models for acute kidney injury. Journal of the American Medical Informatics Association. 2017 Nov 1;24(6):1052–61.
2. Davis SE, Greevy RA, Lasko TA, Walsh CG, Matheny ME. Detection of calibration drift in clinical prediction models to inform model updating. Journal of Biomedical Informatics. 2020 Dec;112:103611.
3. Feng J, Gossmann A, Sahiner B, Pirracchio R. Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees. Journal of the American Medical Informatics Association. 2022 Apr 13;29(5):841–52. 4.

I can add to the paragraph but comment 5 might need a small adjustment to the text too?

Dynamic model calibration

As discussed, models that influence the evolution of its own future input data are at risk of performance deterioration over time due to input data shifts. On the other hand, models can propose poor decisions as a result of the inherent biases found within the original dataset. In both cases, continual learning via model recalibration is required but continual learning remains a challenging paradigm in AI. Recalibration with non-stationary incremental data can lead to catastrophic forgetting when the new data negatively interferes with what the model has already learned[@Parisi2019], or a convergence where the model just predicts its own effect.[@liley2021] Here Pillar 1 (RW-dev) with suitable audit and monitoring via Pillar 2 (ML-Ops) will be required to overcome what would otherwise be a learning process encumbered by regulatory barriers.[@Lee2020]

[Waty-Lilaonitkul]

Hi Steve, Just sent an email to you on the dynamic model calibration section with

1. edited text for latex doc
2. added reference for latex doc