Closed jsv1206 closed 5 months ago
Some useful links for debugging and visualization:
https://github.com/rubin-dp0/tutorial-notebooks/tree/main/DP02_09_Custom_Coadds
https://dp0-2.lsst.io/data-products-dp0-2/index.html https://dm.lsst.org/sdm_schemas/browser/dp02.html
@jsv1206 Do you think the new notebook tutorial will help with resolving the second two points? Should I work on separately verifying the query?
I am not sure if we want to go through this route of making co-add images for each star. Maybe we can leave it up to the PI and give the tutorial notebook reference in the text. I am thinking we can continue with selecting random calexp
images.
The reason, I believe for point 3 i.e. the randomly selected images having different pixel centre etc is because the images are coming from different CCDs which have to be calibrated to get to the same sky map. This is all done in the deepCoadd
pipeline
"To create a custom coadd, only two steps need to be rerun: makeWarp
and assembleCoadd
."
Same goes with point 2. We can leave it to the PI to select/coadd the images they want to show. We can just provide the infrastructure to send flipbook images.
@beckynevin Go ahead and check the query I currently have to get the variable stars and the flux for each variable star. From what I see, it looks fine.
Okay @jsv1206 I'm looking through the query now:
def query_stars(ra_deg, dec_deg, radius_deg, limit):
"""
Query variable stars from dp02_dc2_catalogs.MatchesTruth and dc2_catalogs.TruthSummary
To query more than one star within the circle of search, change = 1 to <= 1
Selecting stars (truth_type=2)
variable (is_variable = 1)
is_pointsource = 1
Input Parameters
----------
ra_deg : ra of the centre of search in degrees
dec_deg : dec of the centre of search in degrees
radius_deg : radius within which to search for
limit : number of variable stars to retireve
Returns
----------
Table of variable stars as pandas dataframe
"""
query = "SELECT mt.id_truth_type, mt.match_objectId, ts.ra, ts.dec "\
"FROM dp02_dc2_catalogs.MatchesTruth AS mt "\
"JOIN dp02_dc2_catalogs.TruthSummary AS ts ON mt.id_truth_type = ts.id_truth_type "\
"WHERE ts.truth_type=2 "\
"AND ts.is_variable = 1 "\
"AND ts.is_pointsource = 1 "\
"AND mt.match_objectId > 1 "\
"AND CONTAINS(POINT('ICRS', ts.ra, ts.dec), CIRCLE('ICRS', "+ str(ra_deg)+", "+str(dec_deg)+", "+str(radius_deg)+")) <= 1 "\
"LIMIT "+str(limit)+" "
results = service.search(query)
variable_stars = results.to_table().to_pandas()
return variable_stars
Here's what I understand from the query: It saves the truth type, objectID, ra, and dec from objects that are matched on the id_truth_type values in the truthsummary and matchestruth tables. The condition it requires is that the truth_type is 2, the objects are variables tars, and pointsources, match_objectID is greater than 1, and it also requires to be within a small radius from your input RA and dec.
I have a couple of questions -
Okay I've been looking through the query_flux() example and it looks good - a good match to tutorial notebook 7b. I was thinking we should add a similar caveat to this in the header or maybe somewhere in the analysis section of the variable stars tutorial. It could reference our thinking that this notebook is not doing the harder step of custom-creating deepcoadds, that can be left to the scientist. We're simply demonstrating how to send a multi-image dataproduct to Zooniverse:
We could also refer them to this notebook (7b) that goes into the Lomb-Scargle periodogram example.
I am not sure if we want to go through this route of making co-add images for each star. Maybe we can leave it up to the PI and give the tutorial notebook reference in the text. I am thinking we can continue with selecting random
calexp
images.The reason, I believe for point 3 i.e. the randomly selected images having different pixel centre etc is because the images are coming from different CCDs which have to be calibrated to get to the same sky map. This is all done in the
deepCoadd
pipeline "To create a custom coadd, only two steps need to be rerun:makeWarp
andassembleCoadd
."
So if you end up going the no-deepcoadd route (only using calexp images), will they just always be misaligned? In other words, is there no way to align them without going through the coadd reduction process?
For the query_stars
function,
AND CONTAINS
part of the query to <= 1. That should query stars within the circleJOIN
tables07b_Variable_Star_Lightcurves.ipynb
truth_type
points to the type of object i.e. star, galaxy or SNE. Refer to the link in 2 for more detailsmatches_objectId
to JOIN
on the Object Table
which has the flux information i.e. we use this ID in the query_flux
functionThe custom coadd tutorial notebook is in DP02_09_Custom_Coadds
GitHub page
https://github.com/rubin-dp0/tutorial-notebooks/tree/main/DP02_09_Custom_Coadds
Tutorial 7b
shows how to query the flux of one variable star and get the phased light curve.
We should add both these notebooks to the reference.
We can mention that we are selecting random MJD in our current notebook but PIs can also create coadded images using 09
tutorial.
I think it is tricky to align calexp images from different CCDs and different exposures. We can dig deeper into makeWarp
and assembleCoadd
tasks in the pipeline which creates the coadds (if we want to take that route). The details are described in https://github.com/rubin-dp0/tutorial-notebooks/blob/main/DP02_09_Custom_Coadds/DP02_09a_Custom_Coadd.ipynb
@beckynevin what to do about the plotting utility functions that are packaged in rubin.citsci
There's some element of randomness to this query - is there a way to get rid of this random element so we can save an output and verify that the brightness change is visible going forward?
Update on this - the individual calexp images look misaligned in WCS because the WCS grid currently centers on each calexp image, even though we are giving it a coordinate. In other words, the image itself is correctly centered, while the WCS is not.
To address some of the above discussion; it is clear that calexp is the image type we want to be sending; deepcoadds would contradict the purpose of what we're trying to do, since deepcoadds smooth out variable objects, which by definition only exist in the difference (calexp) images, not in the deepcoadds.
I'm going to close this issue and open a new one because I am now having the opposite issue; that the images are centered on the same pixel but the WCS is misaligned.