Tutorial: Removing instrumental noise using Pixel-Level Decorrelation (PLD)

[nksaunders]

This notebook is part of a series of tutorials which demonstrate the application of noise-removal techniques to instrumental noise in Kepler/K2 and TESS light curves.

Here, we focus on the Pixel Level Decorrelation (PLD) method. This was first applied to Spitzer time series data by [1], then adapted for K2 data by [2], [3].

This tutorial will provide an introductory understanding of the PLD algorithm and how it was applied to K2. It will also demonstrate with examples how it has been implemented by lightkurve in the PLDCorrector class, and how this class can be used to correct K2 data.

Learning Goals:

• Understand the PLD method
• Understand how to apply the PLDCorrector to light curves
• Provide warnings for common issues and potential biases (thoughts about this @barentsen @mustaric?)

Potential Code Changes:

• Bring lightkurve.PLDCorrector up to speed with the RegressionCorrector and DesignMatrix framework

---

[1] Deming et al. (2015) [2] Luger et al. (2016) [3] Luger et al. (2018)

[mustaric]

I generally like it when tutorials do more than just describe the method. Adding wisdom about both why you might want to do this task and how to recognize when it didn't work as expected makes tutorials extra valuable to the end user.

[barentsen]

This sounds great @nksaunders!

Three thoughts:

1. We'll want to make sure the notebook talks about over-fitting. Of course over-fitting is a big topic and not unique to PLD. Should there be a companion How to avoid over-fitting during systematics correction? tutorial that goes into more detail?

2. Jeff Smith reported that the over-fitting diagnostic tools he's been working on are ready for review (cf. https://github.com/KeplerGO/lightkurve/pull/672). Would it make sense to help review that PR as part of this project? Ideally, Jeff's over/under-fitting diagnostics would also work for PLD!

3. There are a few common questions which always come up when using PLD. It may be fun to answer these in FAQ format. Example questions include:

   • How many pixels should I use?
   • How should I select the pixels to use?
   • How can I speed up the correction?
   • What to do if I get a singular matrix error?

[nksaunders]

@barentsen this notebook is (mostly) ready for review!

[barentsen]

@nksaunders I left a few comments in the notebook. My biggest comment is that it would be nice if we can avoid using BLS in this tutorial -- its use to create a transit mask is a bit confusing. Do you want to have a go at adding a short & sweet create_transit_mask function to Lightkurve?

[nksaunders]

@barentsen I like that idea! I originally had some numpy code to build the mask, but found that using the BLSPeriodogram was much cleaner. I agree that a simple create_transit_mask function would be even better! I'll open a PR.

[barentsen]

@barentsen I left some comments in the Colab notebook. Please ping me if you had a chance to digest them. Thank you!

[barentsen]

I had another pass through this notebook and made a few small changes:

• Removed the Companion Content and Additional Resources sections per the new template.
• Merged the Summary and Introduction sections per the new template.
• I edited the titles to make them a bit longer and more descriptive of the content.
• To motivate the need for Section 2, I added the following paragraph at the start: The success of PLD depends on a number of factors, including the brightness of the object, the choice of pixels used to create the light curve and the noise model, and whether or not there exists a correlation between the instrumental noise and the astrophysical signals. For these reasons, it is important to carefully review the correct operation of the algorithm each time you use it, and tune the optional parameters of the algorithm if necessary.
• I removed references to the companion "how does PLD work" tutorial, so we can proceed with this one and link the other one later if possible.
• Various small edits.

I have yet to look carefully at the FAQ.

@nksaunders Let me know if you were planning to make more changes to this notebook, or if you are happy for me to do a final review before passing on to copy-editing!

[nksaunders]

@barentsen This notebook is now consistent with the current version of lightkurve.PLDCorrector and is ready for review.

[barentsen]

Thanks @nksaunders! This turned into an excellent, helpful, and necessary tutorial in my opinion!

I completed my final review. The notebook is now ready for copy-editing.

Filename: Removing Instrumental Noise from K2 and TESS light curves using Pixel Level Decorrelation (PLD).ipynb

[lglattly]

@nksaunders and @barentsen I finished copy editing this notebook and it was in great shape! I proofread, caught a few typos, replaced some placeholder links with generic text, and made minor copy edits throughout. This notebook is all set on my end and ready to go!

[barentsen]

Thanks @lglattly! :+1:

[barentsen]

This notebook has successfully been merged into spacetelescope/notebooks. Thanks everyone! Closing the issue...