search

GALAHProject / fits-spectrum-format

Description of, and tools to deal with, the GALAH 1D spectrum FITS format
MIT License
0 stars 1 forks source link

Additional extension? #1

Closed SarahMartell closed 9 years ago

SarahMartell commented 9 years ago

Is it reasonable to add the best-fit synthetic template as a fits extension, in addition to its ccf? This would be useful information in early days, but I'm not sure it would continue to be.

andycasey commented 9 years ago

I'm OK with this. Any opposed? /ping @GALAHProject/wg6

andycasey commented 9 years ago

Ping @GALAHProject/wg6 -- strong opinions opposing this should be voiced

sheliak commented 9 years ago

Hi,

A spectrum before the sky subtraction should be one extension, and probably it's sigma as well. If there is a nebula in the field or the weather is bad, it is almost impossible to subtract the sky properly. An automated pipeline can fail quite badly at this and produce spectra that have negative fluxes, for example. It might be better for some analysis pipelines to work on a spectrum without the sky subtraction in some cases, and at least have a chance to identify or go around the problem.

Janez

On 15-04-2015 14:06, Andy Casey wrote:

Ping @GALAHProject/wg6 [1] -- strong opinions opposing this should be voiced

Reply to this email directly or view it on GitHub [2].

*

Links:

[1] https://github.com/orgs/GALAHProject/teams/wg6 [2] https://github.com/GALAHProject/fits-standard/issues/1#issuecomment-93360876

andycasey commented 9 years ago

@sheliak: I have added in the extension for flux before sky subtraction.

After thinking about @SarahMartell's proposal a little more, I think this may be more problematic than it seems. I think it's absolutely important that we can easily access the best-fit model spectra for any analysis, from any code. There is a lot of information there that will help inform us about the reductions and analysis.

For the CCF initial guess, it is just the closest grid point. I am also producing snapshot images of the initial fits (eg #2), and the information of the best-fit spectrum is stored in the image headers. Keeping the best fit CCF spectrum in each image means a lot of replications of grid point spectra (e.g., a lot of disk space), which otherwise could just be extracted afterwards with a couple of lines of code. So I am inclined to leave the best-fit CCF model out of the individual images, and just put examples of how to reproduce the closest CCF model fit in this repository.

If we went the other way and put in the best model fit from any of the analysis codes, that is at least an extra 5+ (e.g., doubling the number of) extensions: CCF, GUESS, Cannon, Theremin, Oracle. Plus, there are many ways to skin a cat: I'd like to experiment with different algorithms in Oracle to see what works best, and when Oracle is doing MCMC there will be many samples from the posterior to consider.

Instead, I would suggest that we require the analysis codes to send back a full filename path of where their best-fit model spectrum exists. That information can be put into the SQL database, and we could even HTTP link it to the exact path (e.g., at AAO FTP or on galah.science).

What do you think about this plan @SarahMartell? It's a balance between extra disk-space/transfer and the ease in accessing the best-fit models.

SarahMartell commented 9 years ago

@andycasey I think you're totally right here.

andycasey commented 9 years ago

OK, I will close this issue for the moment. I want to make plots showing the model residuals as a function of distance to the moon during observations, so I will need to re-create a lot of best model fits then. When I do that I will put the examples in this repository, and we can organise similar examples a bit better into a centralised location later on.