Ugh, Ignore the previous post (accidentally pressed Enter and github has no real delete function). I originally sent these comments to Matteo on slack, and he said the typical review process involves opening a ticket on github, which I assume meant issue. Since I originally sent everything on Slack, it's not particularly well formatted, but here are my comments:
The construction of the EPSF doesn't obviously seem to preferentially choose stars in any way. This is potentially something to look at, as I figure there's a tradeoff between number of stars used and the average SNR of those stars.
-It seems the peak of the distribution of detected stars is at approximately -5.5 mag (instrumental mag, not
exposure time corrected), but there are a fair number of stars brighter than that. Due to the rather undersampled nature of
the PSF, I'm thinking the signal in the wings of the PSF from the faint stars is fairly low, and often in the noise of the
background.
That's especially true since the radius of the constructed EPSF is ~12 pixels, so several FWHM. Not sure it's necessarily critical, but when we do PSF fits for WFC3/UVIS using the empirical PSFs, we only fit the innermost box of 5x5 pixels. For the IR I think it is the same, but I can't remember exactly. Generally we find this does a pretty good job, and allows for measurements of faint stars, without having low signal in the wings cause much issue. I think it may be worth looking at the number/brightness of stars used in the EPSF creation, as well as the size of the created EPSF.
That also has the added benefit of reducing effects on crowding. Obviously, not fitting the full extent of the PSF would require further correction afterwards, but this may be achievable as some form of "aperture correction". How much that aperture correction varies across the FOV and time will likely be the key determinant on whether this is a better strategy or not.
Another note I think is important is the creation of the gridded EPSF, I see that's in the todo list, which is great. For HST we find this to be pretty important as there is substantial variation across the FOV. With the EPSF builder already in place, creating these is probably fairly simple. The GriddedEPSFModel class is easy enough to instantiate, and it looks like there's the skeleton of some of that stuff already there. I know it's not exactly the same here, but we provide the gridded EPSF's for HST instruments, as we know they can be derived from other data and fit with decent results.
-In truth the best results come from the focus dependent EPSFs that we provide, and I have written a subclass of GriddedEPSFModel that handles them, but I wouldn't think that's in the scope of this. The point is that in the future it may be worth looking into providing PSFs already generated from good data, that is parametrized by focus or some other relevant variable.
-I think another useful thing to show may be the outputs of the PSF fitting, as in the catalogs before they go into the CMD's, just so users know what information is available. Comparing fits to groups containing multiple stars vs a single star, or even just commenting on the behavior there could be useful.
-Also showing what the uncertainties look like as a function of flux, and even position (since a single PSF is being used across the field), would be interesting, if only to show as a comparison to gridded model performance
-Showing the subtracted images are nice to show fit quality, but quantifying it with a single number may be good too. With Jay's PSF photometry/methods, we have a q parameter, which is computed as abs(Fit - data)/ (fit flux). Kind of shows how far "off" the data was from the model. This might not be the easiest to implement though.
-The last science related comment I have would be to also give some comment about the astrometric information that comes out of the fits. The benefits of the PSF fitting are often great when looking for accurate positions, and typically better than DAOFind type routines in my experience. A potential way to show this could be to plot the separations between the fit positions of the same stars in the two bands vs mag, to get some idea of astrometric precision. I think that will be very interesting to those looking to measure proper motions/detect microlensing etc.
-Last thing: for readability, putting a lot of the stuff that gets ran twice in a cell into a function could be nice. For instance the detection of stars, the fitting, and the matching of the stars can likely be condensed into functions, and just have lines later that call them. This is also nice because users can copy/paste the relevant functions right out for their own analyses.
Ugh, Ignore the previous post (accidentally pressed Enter and github has no real delete function). I originally sent these comments to Matteo on slack, and he said the typical review process involves opening a ticket on github, which I assume meant issue. Since I originally sent everything on Slack, it's not particularly well formatted, but here are my comments:
The construction of the EPSF doesn't obviously seem to preferentially choose stars in any way. This is potentially something to look at, as I figure there's a tradeoff between number of stars used and the average SNR of those stars.
-It seems the peak of the distribution of detected stars is at approximately -5.5 mag (instrumental mag, not exposure time corrected), but there are a fair number of stars brighter than that. Due to the rather undersampled nature of the PSF, I'm thinking the signal in the wings of the PSF from the faint stars is fairly low, and often in the noise of the background.
That also has the added benefit of reducing effects on crowding. Obviously, not fitting the full extent of the PSF would require further correction afterwards, but this may be achievable as some form of "aperture correction". How much that aperture correction varies across the FOV and time will likely be the key determinant on whether this is a better strategy or not.
Another note I think is important is the creation of the gridded EPSF, I see that's in the todo list, which is great. For HST we find this to be pretty important as there is substantial variation across the FOV. With the EPSF builder already in place, creating these is probably fairly simple. The GriddedEPSFModel class is easy enough to instantiate, and it looks like there's the skeleton of some of that stuff already there. I know it's not exactly the same here, but we provide the gridded EPSF's for HST instruments, as we know they can be derived from other data and fit with decent results. -In truth the best results come from the focus dependent EPSFs that we provide, and I have written a subclass of GriddedEPSFModel that handles them, but I wouldn't think that's in the scope of this. The point is that in the future it may be worth looking into providing PSFs already generated from good data, that is parametrized by focus or some other relevant variable.
-I think another useful thing to show may be the outputs of the PSF fitting, as in the catalogs before they go into the CMD's, just so users know what information is available. Comparing fits to groups containing multiple stars vs a single star, or even just commenting on the behavior there could be useful. -Also showing what the uncertainties look like as a function of flux, and even position (since a single PSF is being used across the field), would be interesting, if only to show as a comparison to gridded model performance -Showing the subtracted images are nice to show fit quality, but quantifying it with a single number may be good too. With Jay's PSF photometry/methods, we have a q parameter, which is computed as abs(Fit - data)/ (fit flux). Kind of shows how far "off" the data was from the model. This might not be the easiest to implement though.
-The last science related comment I have would be to also give some comment about the astrometric information that comes out of the fits. The benefits of the PSF fitting are often great when looking for accurate positions, and typically better than DAOFind type routines in my experience. A potential way to show this could be to plot the separations between the fit positions of the same stars in the two bands vs mag, to get some idea of astrometric precision. I think that will be very interesting to those looking to measure proper motions/detect microlensing etc.
-Last thing: for readability, putting a lot of the stuff that gets ran twice in a cell into a function could be nice. For instance the detection of stars, the fitting, and the matching of the stars can likely be condensed into functions, and just have lines later that call them. This is also nice because users can copy/paste the relevant functions right out for their own analyses.