rmjarvis / Piff

PSFs In the Full FOV. A software package for modeling the point-spread function (PSF) across the full field of view (FOV). Documentation:
http://rmjarvis.github.io/Piff/
Other
58 stars 13 forks source link

Add the size-magnitude selector (née findstars) #127

Closed rmjarvis closed 2 years ago

rmjarvis commented 2 years ago

This PR add a bunch of new functionality, all mostly related to selection based on a size-magnitude diagram.

  1. Selector type = SizeMag. This the fundamental algorithm in the guts of the findstars program. This version is a re-implementation of the concepts involved, not a beat-for-beat copy. The basic idea is to slowly build up a histogram in log(T), adding objects starting with the brightest and moving fainter. The stellar locus is a tall peak of objects all at the same size. We stop when the stellar locus merges into the galaxy locus (at larger size). One trick I copied over is to fit the initial star sizes as a function of (u,v) and subtracting that off, so the stellar locus is at 0. This may not be so useful for a single CCD, but I suspect it may be important if doing selection for a full exposure. (It was vv important for my thesis data on the BTC.)
  2. Selector type = SmallBright. This is the (default) initial star selection for the SizeMag selector. It's not exactly the same as what I had coded up in findstars, but I think it's pretty solid. The idea here is to try to figure out some small number of very high confidence stars and then expand the selection to include objects with nearly the same size as them. The algorithm is to first take the brightest 20% of the detected object. Then take the smallest 20% of these. These are now mostly stars. Maybe all of them. But in case there are some non-stars in the set, pick out the half of them that have the tightest range in log(T). I.e. basically all the same size. Then once these high-confidence stars are identified, let the set expand to objects with about the same size (based on the std(logT) of these stars).
  3. Stats type = SizeMag. This is basically the code that made figure 3 in the Y3 PSF paper. It makes a size - magnitude diagram of the detected objects, initially selected stars, and the stars that survived the outlier rejection during the fitting process. I also added hollow circles for the size of the reconstructed PSF model, which seems like potentially useful information to include there.
  4. Selector type = Properties. This isn't really related to findstars, but it gives a pretty general way to select based on whatever calculation you want from the input properties in the catalog. I think this should provide everything we need to use the Gaia stars as an initial guess for the star selection to use with the SizeMag selector to extend fainter.
  5. sky_file_name. Sorry, this really doesn't have anything to do with findstars, but it was something Theo asked about semi-recently, and it was easy to implement so I did. I use it in the new size_mag.yaml file in the tests directory, which is what I was coding towards.
rmjarvis commented 2 years ago

For reference, here is the size-magnitude diagram that is made by running piffify sizemag.yaml in the tests directory.

sizemag

rmjarvis commented 2 years ago

Thanks Matt!

You may want a test where find stars total craps out with no stars found to test some of the edge cases?

I already have test cases that hit all the lines of code. Including the one you were worried about if the for loop never broke out. I'm quite sure there are cases where it won't do a good job, but I at least think I've tested all the paths through the code to make sure they are valid.

I anticipate adding some test cases from real-world data where this does terribly to try to make it behave better though.

rmjarvis commented 2 years ago

I thought of some pathological inputs to stress test the code a bit. I'm sure there are still lots of failure modes I haven't found, but at least it's hardened to some fairly obvious pathologies now. (e.g. input has 0, 1, 3 stars, input is all galaxies). They fail for these more gracefully now, giving coherent error messages when appropriate rather than e.g. letting numpy raise an opaque exception about finding the median of 0 objects or some such.

I think this is good to go. I'll merge Friday unless anyone wants more time to look it over and give comments.