issue regarding running k2sc -- having trouble reproducing the k2sc team's results

[bethbiller]

We've been using k2sc v 1.0 (downloaded from this repository) with george v 0.3.1 to detrend the lightcurves and remove telescope systematics. Unfortunately, we're having trouble replicating the lightcurves provided by your team here:

https://archive.stsci.edu/prepds/k2sc/

I've attached an example -- this is k2 id 210327027:

also known as 2MASS 0355+11, an L3-L6 brown dwarf observed during campaign 4. In the attached plot, the top panel is the raw PDCSAP flux, the middle panel is our attempt to run k2sc on the PDCSAP flux (clearly something off), and the bottom panel is the result from the lightcurves provided by your team (clearly detrended). All lightcurves have been normalised by the median value, and then had 1.0 subtracted from them (so units should be fractional variation from the median).

We installed k2sc following the directions on this github repository, and with the command line:

k2sc ktwo210327027-c04_llc.fits

where ktwo210327027-c04_llc.fits is the lightcurve file downloaded directly from the MAST archive.

I've put both the input and output (EPIC_210327027_mast.fits) files up on Dropbox, along with the script used to plot the lightcurves (plot_lightcurves.py):

https://www.dropbox.com/sh/u3kgy2j7f1jh4i2/AABMSHSti1d2h6k06nNyLf6ka?dl=0

Any insight you might have into what is going on (and what we might have done wrong) would be greatly appreciated! Thanks in advance!

[benjaminpope]

Hi Beth,

Just to update you, I have been able to reproduce your bug but haven't been able to fix it. Running it either from the command line or from the lightkurve API gives the same bug. I will keep you updated.

@hpparvi do you have any ideas?

All the best,

Ben

[hpparvi]

Hi Beth,

Sorry, I'm on holidays and haven't been checking my mail that frequently. I'll try to find some time to check this within the next day or two.

Cheers, Hannu

[bethbiller]

Hi Ben and Hannu --

Just checking in if there had been any resolution as to what is causing this bug? Thanks in advance!

Best Wishes, Beth

[benjaminpope]

Unfortunately not yet!

On Mon, Mar 2, 2020 at 5:05 AM bethbiller notifications@github.com wrote:

Hi Ben and Hannu --

Just checking in if there had been any resolution as to what is causing this bug? Thanks in advance!

Best Wishes, Beth

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DABN6YFPKFHPEONQSZPILZM3RFOAG5A5CNFSM4KWO7ZCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENOWF4I-23issuecomment-2D593322737&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=zG_oIXjhxT5xnDPHZiCpj0xiQKOvHsAbjwib7X8AotE&s=a2jyvEF3BctUG8NYaVkZ-XwoqR967fl3aDVEAYA_ook&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFLSYRVQQWVLF6RCGWDRFOAG5ANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=zG_oIXjhxT5xnDPHZiCpj0xiQKOvHsAbjwib7X8AotE&s=tOjxM10w8qW6oVpJtaLc6ktksU--BC9wfxXUx_DYZ3c&e= .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[bethbiller]

Hi Ben and Hannu --

Just checking in on this. I have a new research student starting this semester who is interested in running k2sc on K2 brown dwarf lightcurves, so wanted to check if you thought it was possible for this bug to be resolved within the next month or two? If not, I will suggest a different research project for him.

Thanks!

Best Wishes, Beth

[benjaminpope]

Hi Beth,

I'm sorry, I have just been completely slammed and k2sc has been a low priority for me. Hannu, have you had a look at this?

I apologize for being useless, Beth - if this is a research project that you really want to do, I can prioritize this.

All the best,

Ben

On Thu, Sep 17, 2020 at 8:40 PM bethbiller notifications@github.com wrote:

Hi Ben and Hannu --

Just checking in on this. I have a new research student starting this semester who is interested in running k2sc on K2 brown dwarf lightcurves, so wanted to check if you thought it was possible for this bug to be resolved within the next month or two? If not, I will suggest a different research project for him.

Thanks!

Best Wishes, Beth

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-23issuecomment-2D694148929&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=nOziImNtBGzDhdY6YnHmc1vC0mDrqQgbvVpjMfjz1p8&s=aD05M22OBDffntjyFae8w_APUjrlWJvvWcuRMV3-Nu0&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFKSZYBRJVQRCYCPZ2TSGHRSRANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=nOziImNtBGzDhdY6YnHmc1vC0mDrqQgbvVpjMfjz1p8&s=Ch4gdrMztrk4c1Q_uzONjhm7D4o26n8CvnFEJr0R2HE&e= .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[bethbiller]

Hi Ben and Hannu --

So my new student has started and is in fact very keen on working with k2sc on K2 brown dwarf lightcurves. We've tried some of the other detrending software available (Everest, k2sff, etc.) and honestly, only k2sc does a good job detrending the light curves for these faint objects! K2sc will be quite pivotal for this project -- considering this, we're also happy to include the k2sc team on any eventual publication from this work. Thanks!

Best Wishes, Beth

[saigrain]

Hi Beth

Glad to hear it!

I’m sure you’ve seen it already, but just in case, this paper used K2SC for K2 Brown Dwarf LCs too: https://ui.adsabs.harvard.edu/abs/2018ApJ...859..153S/abstract In that case the K2SC analysis was more a sanity check – I ran t for them after they had done their own analysis, to confirm that it gave consistent periods (which it did).

Cheers

Suzanne

-- Professor of Astrophysics University of Oxford Fellow of All Souls College Suzanne.Aigrain@physics.ox.ac.uk +44 1865 273 339 / +44 7549 939 899

From: bethbiller notifications@github.com Reply to: OxES/k2sc reply@reply.github.com Date: Thursday, 24 September 2020 at 10:37 To: OxES/k2sc k2sc@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [OxES/k2sc] issue regarding running k2sc -- having trouble reproducing the k2sc team's results (#24)

Hi Ben and Hannu --

So my new student has started and is in fact very keen on working with k2sc on K2 brown dwarf lightcurves. We've tried some of the other detrending software available (Everest, k2sff, etc.) and honestly, only k2sc does a good job detrending the light curves for these faint objects! K2sc will be quite pivotal for this project -- considering this, we're also happy to include the k2sc team on any eventual publication from this work. Thanks!

Best Wishes, Beth

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/OxES/k2sc/issues/24#issuecomment-698233827, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAAKAOICL47CYTJUG4FVLD3SHMHJZANCNFSM4KWO7ZCA.

[benjaminpope]

Hi @bethbiller,

Sorry to be so lax about this. I am a little overwhelmed at the moment. @hpparvi have you plans to look at this? Or else I can.

Beth - what timescale do you need the fixes by? I have a deadline mid next week after which I may be able to get to this.

All the best,

Ben

[hpparvi]

Hi,

Sorry, I've been busy also for the last months, but will try to find time to fix the issue during the next two weeks.

Cheers, Hannu

[bethbiller]

Hi everyone --

A fix in the next month or so would be great.

Best Wishes, Beth

[benjaminpope]

Hi @hpparvi,

Ok so I'm looking at this and something is clearly wrong. Are the light curves labelled correctly or normalized in some unusual way? I can't actually remember - it was four or five years ago - but just overplotting trtime and trposi on @bethbiller's exact plots, it's clear that the corrections are much smaller than the differences.

Ben

[benjaminpope]

k2sc's standalone version appears to reproduce fairly well Figure 3 from the original paper, correcting EPIC 211748059: https://arxiv.org/abs/1603.09167

[bethbiller]

Hi everyone -- Just checking in on this. While the current standalone version doesn't work for the campaign 4 observation k2 id 210327027, there clearly was an earlier version of k2sc that did provide sensible corrections -- the lightcurve available here: https://archive.stsci.edu/prepds/k2sc/ looks well-corrected. Any chance of accessing that earlier version of k2sc for the additional K2 brown dwarf lightcurves we are looking at?

[benjaminpope]

Hannu - which version of k2sc did the HLSPs use? What made the difference here, I wonder?

On Tue, Nov 10, 2020 at 10:25 PM bethbiller notifications@github.com wrote:

Hi everyone -- Just checking in on this. While the current standalone version doesn't work for the campaign 4 observation k2 id 210327027, there clearly was an earlier version of k2sc that did provide sensible corrections -- the lightcurve available here: https://archive.stsci.edu/prepds/k2sc/ https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.stsci.edu_prepds_k2sc_&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=n-8_nd7yUuSGOwQ2v8aXMIm9rC--0D3yinigNpKj97U&e= looks well-corrected. Any chance of accessing that earlier version of k2sc for the additional K2 brown dwarf lightcurves we are looking at?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-23issuecomment-2D724641467&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=9bsz_GyzSP_NYUKdFhyZs4wJWUBJRPdIaXlqFRfu9Rc&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFKX7T6P3FMN2FAR5P3SPEPIXANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=U5NDHDengck4TRhaZJ_B4jchJIUruzNl56yEQhj9HD0&s=xDez3naCFHhKO1agv2wfKe09YwsSuSr3IdREUwwg93s&e= .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[hpparvi]

Hi @bethbiller and @benjaminpope,

I've finally managed to find a bit of time to look into this, sorry for the long delay. I'm starting to think the issue is not with K2SC, but with the input data.

First, the raw data is a bit weird. The PDCSAP_FLUX median is around -20.7 (yes, negative), while the k2sc version in the MAST has a flux median of 659. The ktwo210327027-c04_llc.fits file has been created 2019-06-26, so it's not the same one as what was used to create the archived K2SC file. I've no idea how to access the older version of the file in MAST, but I'll see if we can find it from somewhere.

So, the difference between the archived K2SC file and the new produced file is most likely because the input photometry is very different. K2SC catches the time variability just fine, but doesn't find any position dependency.

I haven't kept an eye on this, but it's possible that the official K2 pipeline has been updated (@saigrain, @benjaminpope?).

Cheers, Hannu

[bethbiller]

Hi all --

I contacted the STScI helpdesk about accessing previous versions of the 2M 0355 lightcurves. It looks like the version from Autumn 2019 was Data Release 33, here is a link to the Data Release 6 version:

https://archive.stsci.edu/missions/k2/prev_ver/lightcurves/c4/210300000/27000/ktwo210327027-c04_llc.fits

Hannu -- as you suspected, the mean values for the older version of the lightcurve are generally positive. The mean values of the Data Release 6 light curves are ~82 counts for the SAP lightcurve and ~660 counts for the PDCSAP lightcurve. Does this version of the raw lightcurve yield a reasonable result with K2SC?

I've asked the STScI helpdesk as to why the photometry has changed so much between different data releases, will let you know what they say.

Best Wishes, Beth

[samwalker2312]

Hi all,

I'm the student that Beth has been working with on this project and referring to in the above posts. Whilst 2MASS 0355 is clearly a bad example, with its differing data quality between the various K2 data releases, we are still encountering issues when trying to replicate data from the K2SC HLSP archive by running K2SC ourselves. Attached below are a few examples of the issues we're having. You can see that the `detrended' lightcurves that we obtained by running through using the public version of K2SC (labelled as 'our' version) and what has been obtained by your team and placed on the online archive (labelled as 'their version') differs significantly, with obvious variability still present in our data that have been successfully detrended in yours. These lightcurves have all got high fluxes, and have been created using the older versions of the K2 data, before the uniform reprocessing that caused the issues with 2MASS 0355, so we should be using the exact same raw data to create each of these.

It would be great if you could help us understand what's going on here and how to fix it.

Best, Sam

[bethbiller]

Hi Sam -- Could you let the K2SC team know what the K2 id is for these three targets, and which campaign they were each observed in?

[samwalker2312]

Of course, my apologies! These are (from top to bottom): ktwo211776456 (c05), ktwo211962038 (c05), ktwo206135809 (c03) and ktwo210422945 (c04). Hopefully that helps!

[benjaminpope]

Thanks. Sorry for this. I don't know what's happening and we haven't been working on k2sc for a few years now. I worry there is some dependency rot and it will be hard to track down.

What's your python environment/version numbers of everything?

On Wed, Feb 17, 2021 at 10:08 PM samwalker2312 notifications@github.com wrote:

Of course, my apologies! These are (from top to bottom): ktwo211776456 (c05), ktwo211962038 (c05), ktwo206135809 (c03) and ktwo210422945 (c04). Hopefully that helps!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OxES_k2sc_issues_24-23issuecomment-2D780513066&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=f4O78C-kYjz-wWYwhGBuDxnch3oy-Pra2B9e8sVsdkY&s=qWLI9SrABEyAG1-UEY15pjQxXyuVKeMc-s9XxbnvTz0&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ABN6YFM3IQZZ7L4HHAEGLLTS7OWSTANCNFSM4KWO7ZCA&d=DwMCaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=_gjp43gRuTt_LjqJbH0jcZo0ePCz10Y4KegSkx-Ha0A&m=f4O78C-kYjz-wWYwhGBuDxnch3oy-Pra2B9e8sVsdkY&s=dXL4DI41U4MOIpDwcYFw9Qqy_7wZzH7vvRcRc40Zt9s&e= .

-- Dr Benjamin Pope NASA Sagan Fellow Center for Cosmology and Particle Physics // Center for Data Science New York University benjaminpope.github.io

[samwalker2312]

We ran this through using Python 3.7.3, astropy 4.0, George 0.3.1, MPI4Py 3.0.2 (although we didn't use MPI so I imagine that shouldn't be affecting things), NumPy 1.16.2, SciPy 1.2.1, and this current version of K2SC

[hpparvi]

Hi Sam,

Sorry for the slow response. I'll try to have a look at what's going on within the next few days.

Cheers, Hannu