First Release Plan

[dhuppenkothen]

If we really aim for a Jan 31 first release, I think it'd help us stay on track to have a plan for a minimal release, i.e. everything we'd like to have included in our first release.

Here's my list:

• make light curves
• make (& average) power spectra
• fit/sample power spectra with empirical models (power laws, Lorentzians, ...)
• make (& average) cross spectra, compute coherence and lags
• Timmer+Koenig implementation?
• anything else that's essential that I am missing?

It should also include

• documentation for all of the above
• tests for all of the above
• jupyter notebook tutorials for each of the above bullet points

I'd vote for not including too much in the first release, but make sure that all of it is well-documented and well-tested. I already have some code (and a few tests) for points 1,2 and 3, which I could slowly commit over the next couple of weeks, unless @abigailStev thinks we should write it from scratch, or use her version (both of which I'd be fine with, too).

Discuss! :)

[abigailStev]

For the cross spectra bullet point we'll need to compute coherence to get frequency lags and energy lags. I have code for this that we can adapt to suit. Otherwise, let's use your code to start with for power spectra!

[dhuppenkothen]

Do we need the power spectral code to make frequency lags and energy lags and coherence? I don't really know. My power spectral code is almost ready to submit. By the way, I've just made the last commit that I'll just commit straight to the repo. From now on, everything will go in as a pull request. :)

[abigailStev]

We need cross-spectral code for lags and coherence. Cross spectra are mathematically very similar to power spectra, so I'll edit my cross spectral code to be in the same style as your power spectral code.

[abigailStev]

Yay for pull requests 😄🌟

[dhuppenkothen]

Do we want to have a couple of basic (ipython notebook?) tutorials as part of the first release?

[abigailStev]

That's a good idea. I can work on that once your power spectral code is up, since it'll help me learn the ins and outs of your code! Do you have a preferred data set we should use?

[abigailStev]

Sidenote: I think having FITS-file compatibility is really important, as well as (eventual) astroquery support (once astroquery methods are written for heasoft!).

[abigailStev]

I have an ipy nb for Timmer&Koenig that I can adapt to use the light curve class and power spectral code once it's all up there!

[dhuppenkothen]

Do you have a preferred data set we should use?

I don't really. It would be great to have a data set which we can use for multiple tutorials (i.e. power spectra, coherence/cross spectra, fitting Lorentzians, ...), if that happens. Probably some BHB in the low hard state with QPOs? I can probably find one that fits the bill from the GRS 1915+105 data set, but if you have one handy, I'm happy to go with that one.

FYI: I'll be writing a very basic (interactive) tutorial on how power spectral statistics works. I've been working a lot of this out again in order to write tests for the power spectral code, and I'm hoping it'll be a bit clearer than what can be found in the relevant papers (e.g. Michiel's). Also, I want to have it for myself for reference. :) This isn't supposed to be a tutorial on the code itself, however, more of the actual underlying statistics.

[dhuppenkothen]

Sidenote: I think having FITS-file compatibility is really important, as well as (eventual) astroquery support (once astroquery methods are written for heasoft!).

FITS compatibility: depends whether we are having data reading functionality within the package itself. My mental model was that we start with arrays of times/counts or times of arrival that people have already put into python and that they pass to the Lightcurve class (which can then be used in PSDs and stuff). I'd vote for keeping the main code as a library (i.e. without data reading capabilities), but I also vote we'll write some sample scripts and tutorials ("here is how you go from a FITS file to a figure that has the periodogram with Lorentzians fit to it" and similar), in which we'll demo FITS compatibility? Does that sound reasonable?

This would also hopefully allow us to keep our dependencies small (at the moment numpy and scipy).

Not entirely sure what astroquery does. :)

[dhuppenkothen]

I have an ipy nb for Timmer&Koenig that I can adapt to use the light curve class and power spectral code once it's all up there!

That would be awesome to have. I think I added T&K to the general list of "stuff that should be in the package". Not sure yet where it should go yet. Let's figure that out later. The notebook should definitely go in, however.

[abigailStev]

I can probably find one that fits the bill from the GRS 1915+105 data set, but if you have one handy, I'm happy to go with that one.

The only one I really have (and have worked with) is GX 339--4 in the SIMS with the LF QPO. I'm open to either that or GRS 1915.

[abigailStev]

we'll write some sample scripts and tutorials ("here is how you go from a FITS file to a figure that has the periodogram with Lorentzians fit to it" and similar), in which we'll demo FITS compatibility? Does that sound reasonable?

Ok that sounds good to me. FITS i/o is done with astropy, which to me is fine to have as a dependency, but we can include that in just the tutorial ipy notebooks.

[abigailStev]

Not entirely sure what astroquery does. :)

It queries astronomical databases online and returns an astropy table of the data. It's pretty cool, but currently it doesn't support RXTE or XMM. I started working on a heasoft module with some other folks at PyAstro15, but it's gotten pushed to the side since then. So I guess, we should have astropy table support for the lightcurve class to allow for future astroquery compatibility.

[dhuppenkothen]

So I guess, we should have astropy table support for the lightcurve class to allow for future astroquery compatibility.

Can you elaborate a bit on what you actually mean by having both astropy table support and FITS support in the Lightcurve class? I'm not entirely sure what that would look like. Do you want to include a method that does something like read data from a file or from a database query and returns a Lightcurve?

[abigailStev]

Nevermind about the FITS support, as you said we can demonstrate that in an ipy nb tutorial.

astropy table support means that the lightcurve class could receive as input an astropy table of events (or whatever) and make a lightcurve from that.

[abigailStev]

I showed this to @philu74 -- he's on board to be a tester! And he also had a nice idea about saving everything as a data cube: fourier frequency vs energy channel and the reference band, with power spectrum, cross spectrum, and various noise levels.

[dhuppenkothen]

Awesome. Re data cube: that's not a bad idea. It could be some kind of high-level function that computes a bunch of information and saves it in a pandas DataFrame or something similar.