jeff-regier / Celeste.jl

Scalable inference for a generative model of astronomical images
MIT License
183 stars 28 forks source link

Clarification of image gain/calibration parameters, and a new utility script #667

Closed gostevehoward closed 7 years ago

gostevehoward commented 7 years ago

This started as an investigation of #655, but I ended up finally understanding how image calibration and gain works, which was documented here, and I've updated the code in various places to more clearly reflect what I learned. I've updated the parameters used in synthetic field generation to more closely match SDSS. (It may eventually make sense for synthetic fields generation to automatically match the image parameters for the corresponding SDSS field, but that would requires various changes to the pipeline and doesn't seem worthwhile right now.) I also added a script, extract_sdss_image.jl, which writes a FITS file with raw SDSS data in "sky-added" nelec units, which is the form used internally by Celeste, and the form written for synthetic fields. This should aid debugging in the future and helps compare synthetic imagery to real imagery. As an example, here's a side-by-side of real (left) and Synthetic.jl (right) images for RCF 3325/5/110, "r" band:

screenshot from 2017-05-02 15-35-44

Here I've ensured (I think?) that the color scalings are identical in both images. You can see that now they look very similar. (Note this is not the same synthetic imagery as before, because I didn't have quite the correct background/calibration/gain parameters before.)

gostevehoward commented 7 years ago

The new script is documented at the bottom of this page

jeff-regier commented 7 years ago

@kbarbary I'm seeing this error from WCS.jl on travis, but that curl command works fine for me on my local machine. Are you getting this error too? It just started recently.

INFO: Attempting to Create directory /home/travis/.julia/v0.7/WCS/deps/downloads
INFO: Downloading file https://cache.julialang.org/ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-5.13.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   455  100   455    0     0   4739      0 --:--:-- --:--:-- --:--:-- 35000
curl: (35) error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
=================================[ ERROR: WCS ]=================================
LoadError: failed process: Process(`curl -f -o /home/travis/.julia/v0.7/WCS/deps/downloads/wcslib-5.13.tar.bz2 -L https://cache.julialang.org/ftp://ftp.atnf.csiro.au/pub/software/wcslib/wcslib-5.13.tar.bz2`, ProcessExited(35)) [35]
while loading /home/travis/.julia/v0.7/WCS/deps/build.jl, in expression starting on line 39
jeff-regier commented 7 years ago

Looks great!