Astro-Sean / autophot

The AUTOmated Photometry Of Transients (AutoPhOT) is a novel automated pipeline that is designed for rapid, publication-quality photometry of transients.
GNU General Public License v3.0
26 stars 2 forks source link

Template and hotpants subtraction #10

Open Yanshengyu0311 opened 2 years ago

Yanshengyu0311 commented 2 years ago

Hi Sean, When I use the pipeline, I find if the template image quality is not good enough, Hotpants subtraction quality of the output is poor. but if I use Pyzogy, there will be no similar problems. Here is the image subtracted by Hotpants: 图片 The targets are in green circles. The frame of these pictures are locked with WCS

The top left image is the photometry image subtracted by a good template. The top middle image is the photometry image subtracted by a bad template. The top right image is the photometry image without subtraction.

The bottom first image is the bad template in XXX_REDECED/xxx_1/xxx_1_APT_template.fits. The bottom 2nd image is the bad template in XXX/templates/B_template/xxx_0.fits.

In the top-left frame, the result of subtraction is acceptable. In the top-mid frame, there are too many Black circular(which means minus value) spots and the host galaxy region is also much lower than zero. The background is also negative.

The WCS information of the bottom first image is re-written. and the re-written WCS is wrong.

WCS information by scamp has been added in every FITS file including the templates. And the autophot_input["wcs"][‘remove_wcs'] =False autophot_input["wcs"]['force_wcs_redo']=False

When I use the PyZogy, The subtraction is much better.

Another solution besides switching to PyZogy is to modify the parameters of invoking Hotpants in pipelines. And I add my code in template_subtraction.subtract() line 675:

’ template_fwhm=float(list(filter(lambda x :(x[:5]=="FWHM:" and "[ pixels ]" in x ), open(template.split("."+template.split(".")[-1])[0][:-9]+".fits.log","r").readlines()))[0].split()[1]) if template_fwhm > image_fwhm: normalize = "t" toconvolve= "i" hotpants_fwhm=image_fwhm else: normalize = "i" toconvolve= "t" hotpants_fwhm=template_fwhm include_args = [

Input image

          ('-inim',   str(file)),
  # Template Image
          ('-tmplim', str(template)),
  # Output image name
          ('-outim',  str(file.replace(fname_ext,'_subtraction'+fname_ext))),
  # Image lower limits
          ('-il',     str(np.nanmin(check_values_image))),
  # Template lower limits
          ('-tl',     str(np.nanmin(check_values_template))),
  # Template upper limits
          ('-tu',     str(template_max+10)),
  # Image upper limits
          ('-iu',     str(image_max+10)),
  # Image mask
          ('-imi',    str(footprint_loc)),
  # Template mask
          ('-tmi',    str(footprint_loc)),
  # Image gain
          # ('-ig',     str(autophot_input['GAIN'])),
  # Template gain
          # ('-tg',     str(t_header['gain'])),
  # Normalise to image[i]
          ('-n',      normalize),
          ('-c',      toconvolve),                
  # spatial order of kernel variation within region
          ('-ko', '1'),
  # Verbosity - set to as output is sent to file
          ('-v' , ' 0') ,
  # number of each region's stamps in x dimension
          ('-nsx' , '11') ,
  # number of each re5ion's stamps in y dimension
          ('-nsy' , '11'),
  # number of each region's stamps in x dimension
  # RMS threshold forgood centroid in kernel fit
          ('-ft' , '20') ,
  # threshold for sigma clipping statistics
          ('-ssig', '5.0') ,
  # high sigma rejection for bad stamps in kernel fit
          ('-ks' , '3.0'),
  # convolution kernel half width
          ('-r'  , str(2*hotpants_fwhm)) ,
          ('-rss', str(3*hotpants_fwhm)) ,
  # number of centroids to use for each stamp
          ('-nss'  ,str(10)),
          ('-ko'   ,str(0)),
          ('-bgo'  ,str(1)),
          ("-afssc",str(0)),   ]‘

The general idea of the selected parameters is to give -rss, -r, and -n values by comparing image_FWHM and tempale_FWHM. I don't know exactly why these parameters are used, but my colleague told me to do so. And these parameters perform well in my colleague's photometry pipelines.

Here are my questions:

  1. I think the main reason for the bad subtraction is caused by the mistakenly redoing of the template WCS. Do you have any suggestions to turn off this step?

  2. Though I think the Hotpants parameters you selected in the pipeline may be tested lots of times, I still want to try my solution. But when I read the parameter of template_fwhm(by reading the log file of the in XXX/templates/B_template/xxx_0.fits.log, Refer to the code above), it reports a warning: the log file is not close. and this may cause the failure of subtraction through hotpants. I look forward to your advice on a more stable method to invoke the tempale_FWHM.

Astro-Sean commented 2 years ago

Hi!

Thanks for the detailed issue - I really appreciate you informing me of these issues.

There is a lot of things to look over here - Yes those Hotpants parameters were tweaked to be as universal as possible. But probably the best case scenario is to include some flexibility.

I'll look into this and report back.

Seán