mtewes
commented
7 years ago Draft of what I plan to add:
---- WARNING / DISCLAIMER ----
By itself, the use of PyCS does not guarantee realistic time-delay uncertainty estimates. All we have (blindly) demonstrated regarding the quality of PyCS uncertainty estimates assumes a careful generation of the simulated light curves. Keep in mind that the PyCS approach involves setting values or ranges for several parameters. In principle, the impact of these settings has to be assessed on each light curve set, as described in the PyCS paper (Tewes et al. 2013, http://dx.doi.org/10.1051/0004-6361/201220123).
In particular, obtaining uncertainty estimates by running on simulated light curves that all have a same true delay (i.e., setting truetsr=0, in PyCS terminology) can lead to vastly underestimated error bars. It is highly important to use simulations with a range of plausible true time delays to tune and/or verify the accuracy and precision of time-delay estimators. Tests on simulations with only a single true time delay do not probe reliably the quality of a time-delay estimation, as many time-delay estimators are prone to responding unsteadily to the true delay. Do not hesitate to contact us in case of doubts!
Please try to avoid publishing time delays with overly optimistic uncertainty estimates. Seemingly accurate measurements might be propagated into unrealistic cosmological inferences, and harm the community.
Don't tune/simplify the sims to get the tightest possible error bars. Be critical, and show check plots and results on simulations when publishing time delays.