search

HiPERCAM / hipercam

Python package for handling HiPERCAM data
3 stars 4 forks source link

Added "cmax" to psf_reduce.py #80

Closed coclar closed 1 year ago

coclar commented 3 years ago

psf_reduce.py no longer worked due to the cmax key not being included in the results dictionary returned by extractFlux(). This fixes the issue by copying the relevant parts from reduce.py.

trmrsh commented 3 years ago

Ah, I am glad someone is on to this; is this the only problem? I was aware that psf_reduce must be broken after various changes I had made.

trmrsh commented 3 years ago

I will add it in later today after I have dealt with an urgent issue. I guess you have tested that the changes work?

coclar commented 3 years ago

With this fix a psf_reduce call does at least run to completion, and the resulting light curve looks reasonable, but the counts uncertainties seem to be overestimated. Not sure if that is an issue with the script or with my reduction file though, looking into it!

trmrsh commented 3 years ago

one way to over-estimate uncertainties is if you don't subtract any bias an estimate variance from the usual read**2+counts/gain equation. I have never checked psf_reduce's uncertainties in the past

trmrsh commented 3 years ago

Are you still looking at the issue of uncertainties? It occurs to me that it might be better to wait until that is fixed (if there is an issue) before merging, as it will only require an additional merge later otherwise. I am not developing psf_reduce at the moment so there is no risk of complications developing meanwhile

coclar commented 3 years ago

I'm still looking into it. The bias has been subtracted in advance, and I get reasonable uncertainties if I use a normal reduce instead of psf_reduce, so I don't think it's that. The uncertainties come directly from the photutils.BasicPSFPhotometry object used for fitting. I'll keep digging...

coclar commented 3 years ago

Okay, I think I understand what was going wrong. photutils.BasicPSFPhotometry does a least-squares fit, and estimates the flux uncertainties using the variance of the residuals. In my reduce file the gfac parameter (which determines the separation below which stars are jointly fit) was too low, so neighbouring stars were not being subtracted when calculating the residuals and the resulting variance was huge.

An additional problem is that there seems to have been some tracking issues in the observations I'm trying to reduce, so stars are noticeably elongated, and the circular PSF does not fit well. I've whipped up an elliptical version of the Moffat and Gaussian PSFs, but will need some time to get them to work reliably...

trmrsh commented 3 years ago

might not be tracking -- I don't know which data you are looking at, but I noticed some elongation on one night which I am pretty sure is the result of the NTT active optics. But it might not matter one way or the other.

coclar commented 3 years ago

This was hipercam+GTC data. It might not be tracking causing the elongation, but the tracking does also go haywire at some point, so hard to tell! In any case, it seems that elliptical PSF models might be useful for these cases, so I'll try to get this working and update the pull request.

trmrsh commented 3 years ago

that would be of general use I think.