Decide how to handle proper motion

[cwwalter]

Proper motions are currently not handled properly in DM. See:

https://jira.lsstcorp.org/browse/DM-8828

So, I will check on the status of this with the DM team, but if DM does not handle proper motion by the time of DC2 we should consider turning it off. For imSim this would mean setting the proper motion components to zero in the instance catalog. For PhoSim I believe we would pass the ICRS coordinates into the instance catalog instead of the modified ones when we do the CatSim lookup. @danielsf and @johnrpeterson can comment.

[fjaviersanchez]

I agree that if DM doesn't handle proper motion we should deactivate proper motion. That will make our lives easier at the time of matching input and output catalogs.

[cwwalter]

@johnrpeterson Can you comment on the above (how we would turn off proper motion in PhoSim instance catalogs)? Thanks.

[cwwalter]

Hi @johnrpeterson;

It is looking like handling proper motion is not something that is on the DM near term roadmap. Since we think this caused 20mas astrometric errors for us in DC1 we want to understand if we could turn proper motion off so we can decide what to do.

Is this possible in PhoSim, if so, what would the procedure be to make the instance catalogs?

Thanks.

[johnrpeterson]

well this is really at the catalog level. proper motion is either handled by moving the ra/dec at the appropriate time of the observations. the other case is using the movingpoint object which is normally used for high proper motion objects that move significantly in seconds (e.g. asteroids/comets). but for that you would just set it to 0.

is that what you are asking?

john

[danielsf]

We can turn proper motion off in the PhoSim catalogs. However, it is not as simple as "passing in the ICRS coordinates," since that would mean ignoring the precession, nutation, and other motions of the Earth. I do not think we talking about removing those, are we?

[cwwalter]

We can turn proper motion off in the PhoSim catalogs. However, it is not as simple as "passing in the ICRS coordinates," since that would mean ignoring the precession, nutation, and other motions of the Earth. I do not think we talking about removing those, are we?

No.. I don't think so. So, we would need to modify the program that is preparing the instance catalogs for PhoSim and have it not apply any proper motion corrections to the RA and Dec but leave all other modifications to the ICRS coordinates in?

[danielsf]

Yes. We have to replace this method

https://github.com/lsst/sims_catUtils/blob/master/python/lsst/sims/catUtils/mixins/AstrometryMixin.py#L151

with onethat sets pr, pd, and rv to zero.

[danielsf]

(are we also turning parallax off?)

[cwwalter]

(are we also turning parallax off?)

I don't think so.. This is just to address the fact that proper motion isn't handled by DM. I assume parallax is handled somewhere else. @ctslater do you know if this is true?

[SimonKrughoff]

FWIW, this is just a noise term in the astrometric residual, so I'm not convinced we should turn it off.

We don't measure parallax at the moment. Though we could try using these data.

[egawiser]

While its true that one can view uncorrected proper motion as a noise term in the astrometric residual, what took a lot of effort to understand in DC1 was the bias this leads to in astrometry, because of the non-zero average proper motion of the stars used to calibrate it. If we turn off proper motion, it might turn out that parallax becomes the dominant residual, or we might already achieve the rough quality of astrometry that's expected.

[fjaviersanchez]

It seems that this is not trivial and it might arise some other issues if we are not careful...

My main concern is to have a good spatial match between the input and output catalogs. I guess that if we save a truth table with the average positions of the objects over the period that they are going to be observed (so if we are observing certain object during X years get the mean position over those X years and use that as the truth) the match will be easier than for DC1. This creates some overhead at the time of generating the "truth" but it will make things easier at the time of the analysis, and it might be easier than turn off proper motion.

I don't know if this is the best solution and 20 mas is a tiny scale for most probes so I don't think this will be a big deal at the end of the day. @cwwalter @egawiser what do you think?

[egawiser]

In terms of mission-critical items, I worry that this is causing ellipticity due to smearing of the co-added images, and that could affect Weak Lensing. If I understand correctly, each visit gets astrometric calibration against known reference stars. Because the instance catalog gives those reference stars proper motion but DM does not correct for this before fitting the astrometric solution, this causes a shift between the coordinate system defined by the average of the reference stars and the true coordinate system that the unmoving galaxies are in. That should lead to a smearing of the galaxy photometry as time goes on - did you measure ellipticities in DC1? Can we get WL to give us a requirement on the maximum tolerable level of ellipticity added by astrometric systematics for DC2?

[jmeyers314]

Can we get WL to give us a requirement on the maximum tolerable level of ellipticity added by astrometric systematics for DC2?

I did some math on this a long time ago. You can see the DESC WL presentation at: https://confluence.slac.stanford.edu/display/LSSTDESC/Minutes+for+telecon+on+October+6%2C+2014.

The estimate then was that 16 mas RMS misregistration error leads to a systematic comparable to the Y10 statistical power. I'm not sure if there are more recent estimates than this or not...

[rmandelb]

Using the calculation in the HSC WL paper, the rough estimate of the shear calibration bias due to unrecognized astrometric errors is sqrt(sigma_ast^2 + sigma_psf^2)/sigma_psf - 1, where sigma_psf is the Gaussian sigma of the PSF (i.e., modeled as a Gaussian) and sigma_ast is your RMS astrometric error. This assumes the PSF and galaxies are comparable in size. My estimate is that if sigma_ast=0.02 arcsec and PSF FWHM~0.7 arcsec, the shear calibration bias is 0.002. That's likely similar to our requirement for LSST Y10 (i.e., an astrometric error of that size would eat up our entire systematic error budget when it comes to shear calibration bias). But DC2 is going to be much smaller area than LSST, so our requirement will be weaker by quite a bit, and this is probably OK for DC2.

[egawiser]

@rmandelb What happens though if, as occurred in DC1, there is a 0.02 arcsec astrometric bias in the final co-added image? I would naively think that bias doesn't affect WL at all, since it's common to all objects, but that smearing of the galaxies along a particular direction due to imperfect alignment of individual visits would still cause trouble. We can estimate that smearing effect as the delta(average proper motion) between 2023 and 2033 (which I think is less than 0.02 arcsec because the proper motion is mostly caused by an offset between the epoch at which positions were measured and the mid-survey epoch of 2028). If that's accurate, it seems that 0.02 arcsec is an upper limit on the actual rms astrometric error caused by not correcting for proper motion, reinforcing your welcome conclusion above. But does that formula assume a truly random distribution of astrometric errors in two dimensions? This effect is seemingly more coherent than that, which is what worries me.

[cwwalter]

It seems that this is not trivial and it might arise some other issues if we are not careful...

Could we get a clear answer on this point from @danielsf? I still an unclear on whether this is easy or not. In the inSim case is is easy, I'm less clear on PhoSim.

[cwwalter]

What happens though if, as occurred in DC1, there is a 0.02 arcsec astrometric bias in the final co-added image? I would naively think that bias doesn't affect WL at all, since it's common to all objects, but that smearing of the galaxies along a particular direction due to imperfect alignment of individual visits would still cause trouble. We can estimate that smearing effect as the delta(average proper motion) between 2023 and 2033 (which I think is less than 0.02 arcsec because the proper motion is mostly caused by an offset between the epoch at which positions were measured and the mid-survey epoch of 2028). If that's accurate, it seems that 0.02 arcsec is an upper limit on the actual rms astrometric error caused by not correcting for proper motion, reinforcing your welcome conclusion above. But does that formula assume a truly random distribution of astrometric errors in two dimensions? This effect is seemingly more coherent than that, which is what worries me.

Yes, if you take a look at LSSTDESC/SSim_DC1#43 you can see it is quite complicated. We were basically not able to figure it out. There is differential movement because of the MW model and the effect was to get a solution we just couldn't understand. Galaxies and stars acted differently and we effectively just had to give up on understanding what was going on. The algorithm needs to come up with some solution, and it's inputs aren't consistent. So, it's just really hard to understand what it is going to be with all of the movement that is happening.

So, I think if we leave it on, we are just agreeing that we will live with that sort of source dependent shift (with fairly long tails). So on top of ellipticity concerns (which haven't been studied in DC1 yet), that means we won't be able to see or understand any other astrometric residuals less than 20-50 mas. Is that OK? I don't have a good sense of this.

[rmandelb]

@egawiser - The issue of how a bias (as opposed to stochastic offsets) impacts WL depends on things like how is the PSF modeled. For example, let's say you model the per-exposure PSF, and then coadd the per-exposure PSFs to get the coaddPSF model. But when you do that coaddition, you don't know about the coherent smearing along some preferred direction. In that case, the coaddPSF model is wrong, and doesn't describe the fact that everything is smeared a bit along one direction. If we were inferring the PSF from the coadds (which is a bad idea for many reasons) then that would actually fix this one problem (while creating many others) because we'd see that "hey, the stars all have this preferred ellipticity direction" and it would go into the PSF model. But since we don't do that, our PSF model will be wrong and hence it will still cause a problem for WL.

The calculation in my earlier post was for a multiplicative shear calibration bias due the random astrometric offsets, which results in the coaddPSF model being too small compared to the real coadded PSF. In the case of a coherent smearing direction, you'd actually get an additive shear bias correlating with that preferred smearing direction. This could in principle be modeled and marginalized over.

[cwwalter]

Now in downtown Toyama... not sure about internet access today but I wanted to follow up a bit on above to clarify my concern on top of the source dependent shifts we see:

Since we never were able to actually figure out what was going on with the astrometric solution that resulted from not attempting to correct for the motion, I'm afraid there could be other issues which we won't be able to see that could cause other PSF issues like discussed above etc.

We don't actually know how much of the 50mas is due to ignoring the proper motion. So it could be smaller and there may be other issues; we can't tell.

On the other hand, if we include the motion, we can always re-run the stack later if the proper motion support is added to DM. This is a reconstruction not a simulation issue.

[rmandelb]

Chris, if our validation dataset is based on 1 year, then presumably the problems due to neglecting proper motion are less severe, and we can check the astrometry and see if any other major issues pop up? I mean, if we find the same level of problem in 1 year of DC2 data than in 10 years of DC1 data, then that already says it's not proper motion that's the cause. So could we potentially back-burner this question until we analyze the validation dataset?

[egawiser]

That seems like a useful approach. But let me again point out that if the bulk of the proper motion shift is coming from the difference between epoch ~2000 and epoch 2028, we should see almost as much of a shift in 1 year of DC2 data (but much less smearing effect on the PSF, which hasn't as far as I know been noted yet). If instead the nominal coordinates are epoch 2017, the shift should be half as much. @danielsf, can you remind us how the stellar positions are simulated for the instance catalogs?

[cwwalter]

So could we potentially back-burner this question until we analyze the validation dataset?

I think as long as we are willing to regenerate the dataset in anyway, yes this is fine.

But, I think I recall @fjaviersanchez saw this in the individual exposures and it was seen starting in year one. Can you remind us Javier?

I still haven't exactly nailed down the details of exactly what DM is doing with this but I think the fact that the ICRCs are J2000 and the first OpSim observation is Feb 2022 is also relevant.

[cwwalter]

(looks like my similar comment crossed with Eric)

[fjaviersanchez]

@cwwalter yes, that’s correct. I saw this in the individual exposures. Also we have shape measurements on DC1 with hsm (although there was no cosmic shear) so we should be able to test these things.

[danielsf]

@cwwalter I am confident that we can remove the proper motion and just the proper motion in both ImSim and PhoSim.

[connolly]

Just to expand on @danielsf comments. The proper motions are stored in the base catalog used for imsim and phosim (i.e. the database). It would be straightforward to set them (and parallax if wanted) to 0 at the point the database is queried and it should not change anything else in the generation of the phosim inputs

On Thu, Nov 9, 2017 at 12:01 AM, danielsf notifications@github.com wrote:

@cwwalter https://github.com/cwwalter I am confident that we can remove the proper motion and just the proper motion in both ImSim and PhoSim.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/LSSTDESC/DC2_Repo/issues/29#issuecomment-343055778, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDow7J0hG6MKTRuObEgDuUvjUatx5EPks5s0pVWgaJpZM4QL2Xm .

[cwwalter]

would be straightforward to set them (and parallax if wanted) to 0 at the point the database is queried and it should not change anything else in the generation of the phosim inputs

Ah, good point! I see.

[cwwalter]

With the merging of #45, we now have a flag to turn off proper motion when the catalog is made.

I would like to tentatively propose we do that for DC2 since fixing this is not high on the DM priority list see: https://jira.lsstcorp.org/browse/DM-8828 and (since even in year one of the OpSim run we are 22 years out from J2000) this is a noticeable effect that results in a solution we can't understand and is different for galaxies and stars. This ~20mas shift could be hiding other problems we aren't aware of yet and can't check.

[arguing against this positions would be: we can live with this shift, perfect correction is not-realistic, HSC lives with it (although it is smaller there), and eventually we can always rerun DM when they implement the correction].

Let's decide on this in the next day or so.

@egawiser @rmandelb do you want to weigh in given your conversation above?

[cwwalter]

Others with opinions should of course also comment; but let's converge in ~2 days. If no one objects by then, I will close this and ask Jim and Scott to turn the option on to zero out the motion.

[katrinheitmann]

@cwwalter So, ready to close this one? Thanks!

[cwwalter]

OK. For DC2 we will turn off proper motion as implemented by Jim and Scott.

Jim, can you comment whether will also turn off parallax? I am still unclear on whether this is necessary. Then after that is documented we can close this issue.

[jchiang87]

Parallax will still be enabled, i.e., my changes should only affect proper motion. PR #49 has proper motion disabled by default. So I agree that we can close this issue.

[cwwalter]

OK closing. But we should confirm that parallax is handled by DM and what the consequences are if it isn't. @danielsf do you know?

[danielsf]

I do not know

[cwwalter]

@ctslater Can you comment?

[ctslater]

DM does not currently fit parallax, nor is it used for computing the positions of reference catalog stars.

[cwwalter]

Is that bad? :)

[danielsf]

(quick interjection: it is one or two lines of code to turn parallax off the same way Jim turned proper motion off)

[ctslater]

It is not good, but it shares the same root issue as proper motion, so they share a (mostly) common solution. I don't have an easy estimate for the magnitude of the error it would introduce.

[cwwalter]

Well... unlike the proper motion they are not going to keep getting bigger with time as we move away from the year 2000. There is just going to be some offset which switches back and forth and so will average out over time. I assume HSC sees this but it is probably also wiped out by PM. @TallJimbo do you have anything to add?

[cwwalter]

PM = Proper Motion, not Phil Marshall :)

[egawiser]

Nice one! Can anyone give a reason why we shouldn't turn off parallax along with proper motion, given that DM doesn't fit for it?

[TallJimbo]

HSC sees very little of either proper motion or parallax, except perhaps between bands, as we tend to observe depth-first.

[cwwalter]

The only reason I can think of is that there is going to some residual left over from our corrections when we really take data. We are basically getting rid of all of that. So we would be putting in absolutely perfect correction. Parallax would leave something left over, but we don't know how much so this might be a stupid reason.

we tend to observe depth-first.

What does this mean?

[TallJimbo]

we tend to observe depth-first.

What does this mean?

We point at an area of sky on a particular night and basically go to full depth in that area, and then move on to another part of the sky. So for any given area, the baseline for variability is quite small.

I suppose that only applies to the HSC Wide survey, though - we do return regularly to the Deep and UltraDeep fields.

[cwwalter]

Generically following up on my comment above: If we perfectly correct for all motion, we are likely going to be doing better in our astrometric performance than in reality. The reason I wonder if it is a good idea to be "perfect" is if there are in fact correlations with other metrics we want to check.

Will we be making a good test of those other factors if somehow having a perfect astrometric solution makes them too good also? I just have no good feel for this.

[cwwalter]

If this is in fact important, I guess one solution would be to turn everything off and then introduce jitter on the position of all stars drawn from our expected astrometric performance.

[TallJimbo]

Proper motions are definitely a factor in how good the overall astrometry can be, though we're at least hoping that Gaia will basically solve that part of the problem. I don't have any idea how much of a problem parallaxes would be in the absence of proper motions.

But proper motions aren't the only reason astrometry is hard, and I don't think you'll go too far into the realm of unrealistic perfection by disabling them if you make sure you have reasonable astrometric distortions due to the atmosphere and optics.

[cwwalter]

OK thanks Jim. @jchiang87 and @danielsf Can you please also flip off the parallax in the code?