Resolve Supplemental (Methods) Section on Chi_eff Measurement

[farr]

From Ilya:

First of all, the conclusions as expressed in the caption of Fig. 5 and the associated text appear to directly contradict the figure. In the figure, I see that yellow is concentrated at the top and, to a lesser extent, on the right of the figure. Hence, the measurement errors on \chi_eff are smallest when \chi_eff is large, and possibly smaller when q is closer to 1. This is exactly the opposite of what the text says: “\chi_eff is better constrained for systems with low \chi_eff and low mass ratio.”

Secondly, do the inspiral-only waveforms presuppose an abrupt cutoff at the ISCO? If so, then for much of the mass range considered, the inference will be dominated by this artificial cutoff (see arXiv:1404.2382). This can yield unrealistic accuracy and incorrect correlations between mass ratio and spin measurements, and can even change where in the parameter space the measurements are most accurate. I don’t think the results can be trusted if this is the case.

[farr]

More From ilya:

Finally, some more minor points:

• why do these results use cuts on total mass that are inconsistent with two of the three LIGO events observed so far? [I presume the answer is “because that’s what we had available from a different project”, but, frankly, that’s not a very good reason for this paper…]
• what distribution of spin magnitudes and misalignment angles is used, and what priors on these are assumed? what waveform model is used? what (mock) noise is used? what detector network?
• Even ignoring what I think is a misreading of Extended Data Fig. 5, the final sentence of this section is somewhat dubious. E.g., if \chi_eff are low because spin magnitudes are low, then, as shown in Method Section 5, inference on the angles is very difficult…

Given my concerns above, which I think may be difficult to address promptly, I re-iterate my suggestion regarding Section 8 from some time ago. Let’s just add a caveat along these lines:

“The details of how well \chi_eff will be measured depends on the source properties, including the component masses and spins, source orientation and SNR, and detector and network properties which will vary in time [ref to Observing Scenarios document]. Therefore, the predictions regarding the expected number of observations required to reach confident conclusions about the distribution of spin-orbit misalignment angles may change, but we do not expect our qualitative conclusions to be affected.”

[farr]

Ben had some responses and made some changes to text (including fixing the large/small q typo).

[farr]

From Ilya:

Just to be clear: I have no doubt that the actual source parameters, including mass ratios and spins, can impact the accuracy of parameter estimation. And I agree that such a caveat is worth including.

My concern is with the statements we currently make: are they (A) correct and (B) relevant for this particular paper. I agree that relevance is in the eyes of the beholder, so if Will agrees that they are, I have no objection in principle. Regarding correctness, however, there are still some conclusions which, intuitively, I don’t find convincing, and I wonder if they are a consequence of some of the specific choices made, particularly using inspiral-only waveforms with an fixed cutoff. [Aside: was the same frequency cutoff used to estimate the SNR for detectability calculations?]

I do expect that higher \chi_eff will lead to better \chi_eff measurements, as more cycles are spent in the high v/c regime where higher-order pN effects, including spin-orbit coupling, are important. But I don’t see why high q should make it easier to measure \chi_eff; on the contrary, I expect that when q is low, more cycles will be spent near ISCO, and ultimately the test-particle inspiral is the best way to probe the more massive BH’s parameter space. In fact, we did not see the decrease in \chi_eff accuracy at more extreme mass ratios that you find when we analyzed IMRIs with IMR waveforms in arXiv:1511.01431 .

So my preference would be to omit these details, unless we are 100% certain about them, and just include a general caveat.

[farr]

I'm happy to leave the section in (sorry, Ilya). Neither 1511.01431 nor this study correspond precisely to the mass range of interest (this study is too low, 1511.01431 too high); nor do they correspond to the optimal inference (truncated likelihood integral, approximate waveforms, etc). However, I think it is worth reminding people with a concrete example that chi_eff accuracy varies quite a bit over the parameter space, and we may well see large differences in future events.