johannesulf
commented
2 months ago Hi Nasko, thanks for reaching out! This is a great question, and I apologize for somewhat burying the answer in the nautilus paper. In general, there are two sources of uncertainty for log Z: statistical and systematic. "Statistical" here refers to the scatter in log Z between repeated runs with the same settings while "systematic" denotes any potential bias in log Z from the true result over repeated runs. The latter would be non-zero in case nautilus was run using settings that aren't sufficiently "converged."
In practice, the statistical uncertainty on log Z can be very well approximated by 1 / sqrt(N_eff), where N_eff is the effective sample size. That means, by default, nautilus will determine log Z with a statistical uncertainty of around Δ log Z ≈ 1 / sqrt(10,000) = 0.01. This turns out to be substantially smaller than typical uncertainties from traditional nested samplers. Unfortunately, I'm not aware of any method to quantitatively assess the systematic uncertainty from a single run, neither for nautilus nor any other nested sampler.
Fortunately, given the very small statistical uncertainty, even small systematic uncertainties are easy to detect. I suggest varying the number of live points. Additionally, I always recommend discard_exploration=True for publication results.
Does this make sense?
astronasko
Hello, is there a way to assess the logZ uncertainty of a single run?