search

BEAST-Fitting / SEDFitting-outdated

Proto-BEAST code (superseded by the beast)
4 stars 2 forks source link

report output as *sparse* matrices #4

Closed davidwhogg closed 12 years ago

davidwhogg commented 12 years ago

Right now the SED fitting (as far as I can tell) returns a big 4-d block per star. Most of the entries are probably negligible, so it should really just return the non-negligible values in the block. There is a not-wrong way to do this, and we should do it not-wrongly. It involves importance sampling. Ask @davidwhogg for details when you get close to wanting to do this.

karllark commented 12 years ago

I am close to doing this. My nominal idea is just to store 2 1d vectors - elements, and indices to the elements that are above some cutoff. Currently, I would pick something like 1e-50. But I'm a bit worried this might be very IDL specific. I've looked a bit at sparse matrices in the literature (e.g., wikipedia) and they don't seem as clean or as compact for nD as my simple 1D vector idea. The 1D vector idea works as the model grid (in nD) has to be the same between the SED fitting and the cluster fitting (or any other hierarchical model on top of the SED fitting). Will need to have code in this case that ensures the model grid hasn't changed, but this is straightforward to write.

karllark commented 12 years ago

Done. Currently this is done by saving the indices to the 4D full likelihood array and the log likelihood values for elements that are above -40 [TBR] below the maximum log likelihood. Major space savings!

davidwhogg commented 12 years ago

you know what, I am going to re-close this, because I think (looking at the code) that what I would suggest would require a re-factor.