GalSim-developers / GalSim

The modular galaxy image simulation toolkit. Documentation:
http://galsim-developers.github.io/GalSim/
Other
223 stars 105 forks source link

Refinements to the real galaxy calculations #168

Closed rmandelb closed 10 years ago

rmandelb commented 12 years ago

Now that we have a way of simulating galaxies in HST training data, I'm opening this issue as a place to collect suggestions for changes to that part of the code. There are a number of obvious things to be done:

Other suggestions are welcome!

rmandelb commented 11 years ago

This sadly neglected issue related to accuracy of real galaxy calculations needs an update. I'm going to post here a revised list based on the one originally posted here many months ago, plus updates that have occurred since then. Assume that the noise-padding stuff on #238 is done for the purpose of this discussion, since that's nearly true. Issues like optimization are not included, since right now the code is fairly fast and nobody is actively complaining. :) Comments on the plans and/or priority of various items are very welcome:

(1) tests of accuracy of shearing as a function of interpolation kernel: this is super important for the challenge or for any use of the code for the basic shear tests. It's also potentially quite difficult to test in a clear-cut way. Given how important this is, I think it would be nice to discuss how we can do this (pinging the usual suspects: @barnabytprowe , @rmjarvis , and also bothering @gbernstein since it's a natural extension of some of the SHERA vs. SBProfile tests we did before). I have a few ideas, but please let me know if you have more:

(a) Take some reasonably complicated real HST galaxy and maybe two target PSFs and pixel scales (one ground, one space). Simulate the HST galaxy with complications like shear, rotation, magnification, where for each choice of target PSF + shear + rotation + magnification we simulate the galaxy with a range of image padding and interpolant choices. Then, check how the observed images change for the different choices of padding and interpolants. Here we don't have ground truth but we do know which interpolants should do better than others (roughly) so we can at least look for convergence towards a particular answer with better interpolants / more appropriate choice of image padding.

(b) Take one of the HST images and apply to the observed image some operations like shearing, shifting, dilation, rotation. We know how some of the moments should change in each of those cases, so we can look for the correct change in the observed moments.

(c) Make some fake HST data using some analytic profiles / sums of profiles (e.g., bulge + disk) and an OpticalPSF. Use it to simulate sheared lower-resolution data for some target PSF and pixel scale, and compare the resulting images with those we get by directly shearing and convolving the original sums of profiles with the target PSF, then resampling to the target pixel scale.

What else? Is this enough? Depending on the results of these tests, we may end up changing default interpolants / image padding for the RealGalaxy objects, and should certainly update the documentation.

(2) Getting original image / PSF more easily (per Mike's request on this thread from months ago): this should be easy and can take advantage of the InterpolatedImage. It's not super important but it is quick and useful.

barnabytprowe commented 11 years ago

(1) tests of accuracy of shearing as a function of interpolation kernel: this is super important for the challenge or for any use of the code for the basic shear tests. It's also potentially quite difficult to test in a clear-cut way. Given how important this is, I think it would be nice to discuss how we can do this (pinging the usual suspects: @barnabytprowe , @rmjarvis , and also bothering @gbernstein since it's a natural extension of some of the SHERA vs. SBProfile tests we did before). I have a few ideas, but please let me know if you have more

Could I suggest a short telecon to talk through these ideas? Would anyone else be interested? I think they are important and a short discussion (~< 20mins) on the phone might be the most efficient way to plan these. If people agree I'll set up a Doodle, and I guess I'm primarily referring to the names pinged above.

For my part, I think c) is a straightforward and strenuous test (particularly if we go via an InterpolatedImage in the simulation of fake HST data). b) looks quick and very sensible, and a) is a bit sprawling but does have to be done. I do think these will be enough: I also think we might not get them all to pass right away. I cannot think of any more to add, but I've been thinking about it.

rmandelb commented 11 years ago

On Jan 28, 2013, at 7:47 AM, Barnaby Rowe notifications@github.com wrote:

(1) tests of accuracy of shearing as a function of interpolation kernel: this is super important for the challenge or for any use of the code for the basic shear tests. It's also potentially quite difficult to test in a clear-cut way. Given how important this is, I think it would be nice to discuss how we can do this (pinging the usual suspects: @barnabytprowe , @rmjarvis , and also bothering @gbernstein since it's a natural extension of some of the SHERA vs. SBProfile tests we did before). I have a few ideas, but please let me know if you have more

Could I suggest a short telecon to talk through these ideas? Would anyone else be interested? I think they are important and a short discussion (~< 20mins) on the phone might be the most efficient way to plan these. If people agree I'll set up a Doodle, and I guess I'm primarily referring to the names pinged above.

For my part, I think c) is a straightforward and strenuous test (particularly if we go via an InterpolatedImage in the simulation of fake HST data). b) looks quick and very sensible, and a) is a bit sprawling but does have to be done. I do think these will be enough: I also think we might not get them all to pass right away. I cannot think of any more to add, but I've been thinking about it.

Hmm, what else do you think there is to discuss? :)

I mean, if it seems like a straightforward case of "these all must be done" then perhaps we should do them, and then discuss…? (for some value of "we")

barnabytprowe commented 11 years ago

I was hoping that perhaps we could try and get some others on the line too beyond you and I (e.g. a couple of the guys from UCL, even) who share the same interests in the validation of these features, and use the time to informally allocate tasks (which when everyone can speak takes a few minutes only). I was maybe also hoping that it would form a motivational aid, and that the event of talking (even briefly) about this stuff would get people to take a look properly at the proposals would maybe generate new ideas.

barnabytprowe commented 11 years ago

That was the rationale anyhow.

gbernstein commented 11 years ago

I could join a call about validating the simulation codes.

Gary

On Jan 28, 2013, at 11:35 PM, rmandelb notifications@github.com wrote:

Could I suggest a short telecon to talk through these ideas? Would anyone else be interested? I think they are important and a short discussion (~< 20mins) on the phone might be the most efficient way to plan these. If people agree I'll set up a Doodle, and I guess I'm primarily referring to the names pinged above.

rmandelb commented 11 years ago

Sure.

On Jan 29, 2013, at 4:59 AM, Barnaby Rowe notifications@github.com wrote:

I was hoping that perhaps we could try and get some others on the line too beyond you and I (e.g. a couple of the guys from UCL, even) who share the same interests in the validation of these features, and using the time to informally allocate tasks which when everyone can speak takes a few minutes only. I was maybe also hoping that it would form a motivational aid, and that the event of talking (even briefly) about this stuff would get people to take a look properly at the proposals would maybe generate new ideas.

— Reply to this email directly or view it on GitHub.


Rachel Mandelbaum rmandelb@andrew.cmu.edu http://www.andrew.cmu.edu/~rmandelb/

barnabytprowe commented 11 years ago

Excellent. Your perspective would be very welcome Gary!

@tomaszkacprzak, @lvoigt , & @michaelhirsch , would you be able to join a short call to discuss these tests? I know you're very interested in making sure that the real galaxy calculations in GalSim are validated in terms of their precision. These tests, if we get them right, should allow us to nail the issue.

I've set up the following Doodle poll for a 30 min (max) discussion in the next week: http://www.doodle.com/q67bhacaiv3zud7m

Please enter your availability (or let me know if I need to add more time slots). Thanks all!

rmandelb commented 11 years ago

@msimet , I wanted to bring this topic to your attention as well (for once you're done with #306). If this sounds interesting to you, it might be most useful if you join the phone call yourself.

barnabytprowe commented 11 years ago

OK guys, Tuesday at 5pm GMT / 12 noon EST looks like a clear winner, but there are also some possible (if not ideal) times on Monday. Thanks for responding so promptly everyone!

Let's pencil in the Tuesday time, pending confirmation first thing tomorrow am.

barnabytprowe commented 11 years ago

I think we should assume the Tuesday date and time is now set, as we've not heard from any more potential participants. Look forward to speaking at Tuesday at 5pm GMT / 12 noon EST.

rmandelb commented 11 years ago

All - telecon reminder, talk to you in a few hours... Barney - are we doing this on skype? Do you know everyone's contact info? I think it would be preferable if you could initiate the call, if it's not a problem for you. I'm teaching until 5 minutes before the telecon is supposed to start, so I could be a few minutes late.

barnabytprowe commented 11 years ago

I was planning Skype, as I think all are Skype users. The only contact info I'm lacking is for Melanie: would you mind adding me as a contact Melanie? I'm on "barnaby_rowe" and I'll keep an eye out to approve the request... Very happy to lead the call.

msimet commented 11 years ago

It's not letting me add you (says you don't exist)--shall we try the other way around? I'm "msimet".

On Tue, Feb 5, 2013 at 9:39 AM, Barnaby Rowe notifications@github.comwrote:

I was planning Skype, as I think all are Skype users. The only contact info I'm lacking is for Melanie: would you mind adding me as a contact Melanie? I'm on "barnaby_rowe" and I'll keep an eye out to approve the request... Very happy to lead the call.

— Reply to this email directly or view it on GitHubhttps://github.com/GalSim-developers/GalSim/issues/168#issuecomment-13131960..

barnabytprowe commented 11 years ago

That's weird. You also didn't show up using the search term "msimet", but when I searched for Melanie Simet "msimet" came up...

Barnaby Rowe Postdoctoral Research Associate

Department of Physics & Astronomy University College London Gower Street London WC1E 6BT United Kingdom

On 5 Feb 2013, at 15:48, msimet wrote:

It's not letting me add you (says you don't exist)--shall we try the other way around? I'm "msimet".

On Tue, Feb 5, 2013 at 9:39 AM, Barnaby Rowe notifications@github.comwrote:

I was planning Skype, as I think all are Skype users. The only contact info I'm lacking is for Melanie: would you mind adding me as a contact Melanie? I'm on "barnaby_rowe" and I'll keep an eye out to approve the request... Very happy to lead the call.

— Reply to this email directly or view it on GitHubhttps://github.com/GalSim-developers/GalSim/issues/168#issuecomment-13131960..

— Reply to this email directly or view it on GitHub.

rmjarvis commented 10 years ago

I think this issue might be closable?

rmandelb commented 10 years ago

Agree.

rmandelb commented 10 years ago

Closing #168.