Test 2.1.0 v2

[r-pascua]

2.1.0 -- Testing abscal, smooth_cal, and pspec with Foregrounds + EoR

Power spectra from absolutely calibrated visibilities agree with the true power spectra to better than 1% accuracy. Smoothing calibration solutions tends to introduce a slight bias in the final power spectrum estimate (typically at the percent level, but up to about 10% at the band edges); it also introduces yet-unexplained structure in the residuals at low-k.

Contributors:

• @r-pascua
• @zacharymartinot

Some questions

• [x] All tests passed
• [x] I used/followed the notebook template
• [x] All HERA software has full git commit hash referenced
• [x] All data used is explicitly referenced

Steps Forward

A suggested list of condensed automatic tests arising from this PR, and relevant repos to include them in:

A suggested list of follow-up validation tests which may not already appear on the validation path:

• Investigate the accuracy of power spectrum estimates from smoothed calibration solutions; in particular, figure out what is causing the unusual structure at low-k. See later comment for details.
• Investigate how well abscal performs on noisy data when calibrating to a model without EoR.

@jaguirre @steven-murray @jsdillon

[review-notebook-app[bot]]

Check out this pull request on 

You'll be able to see Jupyter notebook diff and discuss changes. Powered by ReviewNB.

[r-pascua]

Details regarding smooth_cal follow-up test:

To recap, smoothing calibration solutions introduces unusual (somewhat wavy) structure in the power spectrum residuals at low-k, and this structure looks to appear at delays around roughly twice the smoothing scale used. To diagnose this issue, some proposed tests are as follows:

• Use gains with simple delay structure (e.g. a sine wave) to uncalibrate the visibilities and run the test again.
• Using the same gains as in this test, re-do the smoothing step with different smoothing scales (suggested 50 ns and 1000 ns as a first attempt) and see how the residuals change.

Important to note is that the structure remains after performing an incoherent average over all times and another average over all baseline pairs. This may suggest that the same structure is appearing for every baseline at each time, but it is hard to say given how bright the foregrounds are at the k-modes of interest. So, it may be worthwhile to see if the structure in the residuals is baseline- or time-dependent. Alternatively, the suggested tests may be performed on a single file (~10 integrations), with all other parts of the analysis left unchanged.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:30Z ----------------------------------------------------------------

Can you insert a link to something explaining the Gibbs phenomenon? Also, can we make it clear if we expect smoothing to do better than not smoothing? i.e. is there anything in the applied gains that the smoothing should fix, or are we merely hoping it to be a roughly identity operation?

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:31Z ----------------------------------------------------------------

I'm going to get you to move the "Last executed" line into a code cell and have it just print, because it doesn't seem to render well.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:32Z ----------------------------------------------------------------

It shouldn't change anything, but I would prefer to use the -1 index if you want to transform over the last axis, in case data is more than 2D (which it shouldn't be in this case...)

r-pascua commented on 2020-02-06T20:39:24Z ----------------------------------------------------------------

I've updated it to assume the frequency axis is the last axis, but now there's the option to specify a different axis. Had to add a few lines to correctly handle reshaping the taper so that the element-wise product is calculated correctly, but I think it should work correctly.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:33Z ----------------------------------------------------------------

Not something you have to fix, but wondering why you save this to disk rather than just having it as a function that runs after you read in data further down? Surely it's quicker to to inflate data that's already read into memory than it is to read the full inflated data from disk?

r-pascua commented on 2020-02-06T20:42:09Z ----------------------------------------------------------------

I forget exactly what the problem was, but I remember that there was an issue with one of the metadata arrays (maybe the baseline_array?) not updating correctly after inflating by redundancy, but that the problem went away if the data was saved and re-loaded.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:35Z ----------------------------------------------------------------

You ask a number of questions in this cell input -- are they resolved? Do they matter for this run?

r-pascua commented on 2020-02-06T20:43:16Z ----------------------------------------------------------------

Oh, I forgot about that! I don't think they matter, so I'll remove them.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:31Z ----------------------------------------------------------------

I'm going to get you to move the "Last executed" line into a code cell and have it just print, because it doesn't seem to render well.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:32Z ----------------------------------------------------------------

It shouldn't change anything, but I would prefer to use the -1 index if you want to transform over the last axis, in case data is more than 2D (which it shouldn't be in this case...)

r-pascua commented on 2020-02-06T20:39:24Z ----------------------------------------------------------------

I've updated it to assume the frequency axis is the last axis, but now there's the option to specify a different axis. Had to add a few lines to correctly handle reshaping the taper so that the element-wise product is calculated correctly, but I think it should work correctly.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:33Z ----------------------------------------------------------------

Not something you have to fix, but wondering why you save this to disk rather than just having it as a function that runs after you read in data further down? Surely it's quicker to to inflate data that's already read into memory than it is to read the full inflated data from disk?

r-pascua commented on 2020-02-06T20:42:09Z ----------------------------------------------------------------

I forget exactly what the problem was, but I remember that there was an issue with one of the metadata arrays (maybe the baseline_array?) not updating correctly after inflating by redundancy, but that the problem went away if the data was saved and re-loaded.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:35Z ----------------------------------------------------------------

You ask a number of questions in this cell input -- are they resolved? Do they matter for this run?

r-pascua commented on 2020-02-06T20:43:16Z ----------------------------------------------------------------

Oh, I forgot about that! I don't think they matter, so I'll remove them.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:32Z ----------------------------------------------------------------

It shouldn't change anything, but I would prefer to use the -1 index if you want to transform over the last axis, in case data is more than 2D (which it shouldn't be in this case...)

r-pascua commented on 2020-02-06T20:39:24Z ----------------------------------------------------------------

I've updated it to assume the frequency axis is the last axis, but now there's the option to specify a different axis. Had to add a few lines to correctly handle reshaping the taper so that the element-wise product is calculated correctly, but I think it should work correctly.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:33Z ----------------------------------------------------------------

Not something you have to fix, but wondering why you save this to disk rather than just having it as a function that runs after you read in data further down? Surely it's quicker to to inflate data that's already read into memory than it is to read the full inflated data from disk?

r-pascua commented on 2020-02-06T20:42:09Z ----------------------------------------------------------------

I forget exactly what the problem was, but I remember that there was an issue with one of the metadata arrays (maybe the baseline_array?) not updating correctly after inflating by redundancy, but that the problem went away if the data was saved and re-loaded.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:35Z ----------------------------------------------------------------

You ask a number of questions in this cell input -- are they resolved? Do they matter for this run?

r-pascua commented on 2020-02-06T20:43:16Z ----------------------------------------------------------------

Oh, I forgot about that! I don't think they matter, so I'll remove them.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:33Z ----------------------------------------------------------------

Not something you have to fix, but wondering why you save this to disk rather than just having it as a function that runs after you read in data further down? Surely it's quicker to to inflate data that's already read into memory than it is to read the full inflated data from disk?

r-pascua commented on 2020-02-06T20:42:09Z ----------------------------------------------------------------

I forget exactly what the problem was, but I remember that there was an issue with one of the metadata arrays (maybe the baseline_array?) not updating correctly after inflating by redundancy, but that the problem went away if the data was saved and re-loaded.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:35Z ----------------------------------------------------------------

You ask a number of questions in this cell input -- are they resolved? Do they matter for this run?

r-pascua commented on 2020-02-06T20:43:16Z ----------------------------------------------------------------

Oh, I forgot about that! I don't think they matter, so I'll remove them.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:35Z ----------------------------------------------------------------

You ask a number of questions in this cell input -- are they resolved? Do they matter for this run?

r-pascua commented on 2020-02-06T20:43:16Z ----------------------------------------------------------------

Oh, I forgot about that! I don't think they matter, so I'll remove them.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:36Z ----------------------------------------------------------------

Can probably remove the XXX what should go in the new history string comment as I'm not sure we care :-)

r-pascua commented on 2020-02-06T20:44:12Z ----------------------------------------------------------------

Good catch!

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:37Z ----------------------------------------------------------------

Might be useful to add a few more comments here regarding the characteristics of the smoothed gains, eg. that they introduce largest errors on the band edges, and seemingly an oscillatory structure in frequency.

Maybe not, looks like you do that in the next cell :-)

r-pascua commented on 2020-02-06T20:50:17Z ----------------------------------------------------------------

Added an extra comment in the caption anyway.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:38Z ----------------------------------------------------------------

Any comment on the weird glitch at the ~1.5 hour mark?

r-pascua commented on 2020-02-06T20:46:08Z ----------------------------------------------------------------

I'll add a comment that although the glitch is an unusual feature, it's at the same scale as the rest of the errors and so doesn't really matter.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:39Z ----------------------------------------------------------------

Note that this cell works in python 3.7+ but may not work in previous versions, as dicts are not ordered (so the labels could be mixed up compared to the globs).

r-pascua commented on 2020-02-06T20:55:50Z ----------------------------------------------------------------

Ah, right--that feature wasn't implemented until Py3.6... I'll change it to an OrderedDict for good measure

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:40Z ----------------------------------------------------------------

Maybe note that *in particular* it's not the calibration that's introducing the leakage to high-k, because the "true" power has the same property.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:42Z ----------------------------------------------------------------

Can you make a note above this plot as to what you're going to plot and why? I know it's in the caption below, but on reading, the plot kind of just "appears" without explanation.

r-pascua commented on 2020-02-07T19:55:50Z ----------------------------------------------------------------

These plots have been removed.

review-notebook-app[bot] commented 4 years ago

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[review-notebook-app[bot]]

View / edit / reply to this conversation on ReviewNB (backstory for this conversation format).

steven-murray commented on 2020-02-06T16:53:43Z ----------------------------------------------------------------

I'm not convinced that P_foreground/P_true is a particularly good metric of uncertainty, and I know this is something we chose somewhat arbitrarily, and will do better on in the future. However, it could use a *little bit* of justification here (even handwavy).

• © Githubissues.
• Githubissues is a development platform for aggregating issues.