search

barnabytprowe / great3-public

Public repository for the Third Gravitational Lensing Accuracy Testing Challenge
BSD 3-Clause "New" or "Revised" License
9 stars 11 forks source link

Regarding the variable_PSF/ground/constant branch #17

Open betajzhang opened 10 years ago

betajzhang commented 10 years ago

Hi Guys,

I've worked on the variable_PSF/ground/constant branch for quite a while. The interesting thing is that I got roughly the same shear values for different sub-fields in the same field (there are 20 of them in each field). I tried two different ways of reconstructing the PSF field myself, and I also tried to use the PSF reconstructed by the code provided by GREAT3. All of these methods yielded very similar answers consistently, but scored "0". Now I suspect that I downloaded the wrong set of data. Have people ever had this situation? Had the data in the "variable_PSF/ground/constant" branch been updated before? Thanks in advance for your time!!

Best, Jun

rmandelb commented 10 years ago

Hi Jun -

You're right that something seems odd here. The variable PSF branches were updated quite a few months ago, not recently. Do you happen to recall when you downloaded this dataset?

I have a few thoughts: (1) Did you compare the md5sum for the entire tar ball with what is on the download site? That is one way for participants to ensure they have the right dataset.

(2) If you've already unzipped and untarred it, then another option would be for you to e-mail me a single randomly-chosen catalog or image file and I will compare with what I have here to see if they are the same. (However, note that I am going away after 7pm Eastern time on Sunday, and will be completely online for 2 days after that.)

(3) Usually a score of 0 means something is actually wrong with the format of the file being submitted. Did you run your submission through the submission checking script at https://github.com/barnabytprowe/great3-public/blob/master/presubmission_script/check_submission.py ? Running the presubmission checker would tell you if there is some formatting issue that our server is having trouble handling when it evaluates your submission. I doubt this is the problem considering your success in submitting to other branches, but I wanted to mention just in case.

I hope that helps!

rmandelb commented 10 years ago

Actually, Jun, given that I'm going away in a few hours, I thought it best to send you some things you can compare with. I've gone into my unzipped version of the variable_psf/ground/constant dataset, and computed md5sums for the image and catalog files in subfield 137 (which I chose semi-randomly). You can use these to check against your files:

b9bea8261f31944c6767948ad42cfbdb galaxy_catalog-137.fits d1d794a8a95b8b7ab7975a8ad2f57a34 galaxy_catalog-137.txt ea8cebff20ffeb493e0d757548823459 image-137-0.fits 0350efef4b6eef8325e4796d2c62ee8b star_catalog-137.fits a60f42f44bae06f1a58b03d3f387d162 star_catalog-137.txt 75f3ef8fe61224c82b8ecd2d0a138307 starfield_image-137-0.fits

If anything is unclear, perhaps someone else can help you while I'm offline. ( @msimet , if you want to know where my unzipped copy of this branch lives, it's on coma, /physics/rmandelb/great3-v11/public/variable_psf/ground/constant - so if any other questions about versions come up, could you please help Jun out? I'm not sure if Barney has an unzipped version of this branch somewhere.)

msimet commented 10 years ago

Noted, thanks--I'm happy to look into things if necessary.

betajzhang commented 10 years ago

Hi Rachel,

Thank you so much for the information!! I will try to compare myself using your data.

Best, Jun

----- 原始邮件 ----- 发件人: "rmandelb" notifications@github.com 收件人: "barnabytprowe/great3-public" great3-public@noreply.github.com 抄送: "betajzhang" betajzhang@sjtu.edu.cn 发送时间: 星期一, 2014年 4 月 21日 上午 1:31:06 主题: Re: [great3-public] Regarding the variable_PSF/ground/constant branch (#17)

Actually, Jun, given that I'm going away in a few hours, I thought it best to send you some things you can compare with. I've gone into my unzipped version of the variable_psf/ground/constant dataset, and computed md5sums for the image and catalog files in subfield 137 (which I chose semi-randomly). You can use these to check against your files:

b9bea8261f31944c6767948ad42cfbdb galaxy_catalog-137.fits d1d794a8a95b8b7ab7975a8ad2f57a34 galaxy_catalog-137.txt ea8cebff20ffeb493e0d757548823459 image-137-0.fits 0350efef4b6eef8325e4796d2c62ee8b star_catalog-137.fits a60f42f44bae06f1a58b03d3f387d162 star_catalog-137.txt 75f3ef8fe61224c82b8ecd2d0a138307 starfield_image-137-0.fits

If anything is unclear, perhaps someone else can help you while I'm offline. ( @msimet , if you want to know where my unzipped copy of this branch lives, it's on coma, /physics/rmandelb/great3-v11/public/variable_psf/ground/constant - so if any other questions about versions come up, could you please help Jun out? I'm not sure if Barney has an unzipped version of this branch somewhere.)

Reply to this email directly or view it on GitHub: https://github.com/barnabytprowe/great3-public/issues/17#issuecomment-40900021

betajzhang commented 10 years ago

Hi Rachel,

My files did pass the md5sum check. I will try to figure out the problem. Thanks again for your time!

Best, Jun

----- 原始邮件 ----- 发件人: "rmandelb" notifications@github.com 收件人: "barnabytprowe/great3-public" great3-public@noreply.github.com 抄送: "betajzhang" betajzhang@sjtu.edu.cn 发送时间: 星期一, 2014年 4 月 21日 上午 1:31:06 主题: Re: [great3-public] Regarding the variable_PSF/ground/constant branch (#17)

Actually, Jun, given that I'm going away in a few hours, I thought it best to send you some things you can compare with. I've gone into my unzipped version of the variable_psf/ground/constant dataset, and computed md5sums for the image and catalog files in subfield 137 (which I chose semi-randomly). You can use these to check against your files:

b9bea8261f31944c6767948ad42cfbdb galaxy_catalog-137.fits d1d794a8a95b8b7ab7975a8ad2f57a34 galaxy_catalog-137.txt ea8cebff20ffeb493e0d757548823459 image-137-0.fits 0350efef4b6eef8325e4796d2c62ee8b star_catalog-137.fits a60f42f44bae06f1a58b03d3f387d162 star_catalog-137.txt 75f3ef8fe61224c82b8ecd2d0a138307 starfield_image-137-0.fits

If anything is unclear, perhaps someone else can help you while I'm offline. ( @msimet , if you want to know where my unzipped copy of this branch lives, it's on coma, /physics/rmandelb/great3-v11/public/variable_psf/ground/constant - so if any other questions about versions come up, could you please help Jun out? I'm not sure if Barney has an unzipped version of this branch somewhere.)

Reply to this email directly or view it on GitHub: https://github.com/barnabytprowe/great3-public/issues/17#issuecomment-40900021

rmandelb commented 10 years ago

Hmmm, interesting, but I'm sorry there's no quick fix. Please let me know if you have any other questions that could help answer this problem...

barnabytprowe commented 10 years ago

Hi Jun,

I just wanted to flag up #18 , which is possibly relevant, and offer my sincere apologies if this problem in the submission checker script contributed to your not being able to successfully submit. I hope that you will nonetheless consider submitting to the post-challenge leaderboards when these go up (soon)!

betajzhang commented 10 years ago

Hi Barnaby,

Thanks a lot! My bug is a little subtle though. It will be very helpful if from somewhere, I can download both the galaxy images and their corresponding PSF images (at the galaxy positions) for the branch of "variable-PSF_ground_constant". It is because I am still not sure if I am using the right data, and if I ran the "psf_models.py" code correctly for reconstructing the PSF field. Would it be a lot trouble for you to generate the PSF images at the positions of the galaxies, and upload them so that I can download? If so, I can quickly re-run my code (the part without PSF reconstruction), and submit my results. Thank you very much for your time!

Best, Jun

----- 原始邮件 ----- 发件人: "Barnaby Rowe" notifications@github.com 收件人: "barnabytprowe/great3-public" great3-public@noreply.github.com 抄送: "betajzhang" betajzhang@sjtu.edu.cn 发送时间: 星期四, 2014年 5 月 01日 下午 9:15:18 主题: Re: [great3-public] Regarding the variable_PSF/ground/constant branch (#17)

Hi Jun,

I just wanted to flag up #18 , which is possibly relevant, and offer my sincere apologies if this problem in the submission checker script contributed to your not being able to successfully submit. I hope that you will nonetheless consider submitting to the post-challenge leaderboards when these go up (soon)!

Reply to this email directly or view it on GitHub: https://github.com/barnabytprowe/great3-public/issues/17#issuecomment-41908671

barnabytprowe commented 10 years ago

Hi Jun,

OK, well I'm glad that it seems the submission_checker.py script wasn't the issue. Your request is going to be difficult to meet in the near future, though, as it would require some code preparation and our simulation generation infrastructure would need to be set up again. Right now we're pretty busy with preparing the GREAT3 code for public release (see below) and analyzing the GREAT3 results in advance of the challenge.

However, in the near future we will be releasing the GREAT3 simulation code as well as the truth data, so you should be able to edit that and generate the images that you want on your own computing network, without needing to download the images. We'll keep everyone updated about this once it's ready.

betajzhang commented 10 years ago

Hi Barnaby,

Sure. Thanks. I understand that you must be very busy right now. I will return to my issue in the near future.

Best, Jun

----- 原始邮件 ----- 发件人: "Barnaby Rowe" notifications@github.com 收件人: "barnabytprowe/great3-public" great3-public@noreply.github.com 抄送: "betajzhang" betajzhang@sjtu.edu.cn 发送时间: 星期四, 2014年 5 月 01日 下午 10:19:17 主题: Re: [great3-public] Regarding the variable_PSF/ground/constant branch (#17)

Hi Jun,

OK, well I'm glad that it seems the submission_checker.py script wasn't the issue. Your request is going to be difficult to meet in the near future, though, as it would require some code preparation and our simulation generation infrastructure would need to be set up again. Right now we're pretty busy with preparing the GREAT3 code for public release (see below) and analyzing the GREAT3 results in advance of the challenge.

However, in the near future we will be releasing the GREAT3 simulation code as well as the truth data, so you should be able to edit that and generate the images that you want on your own computing network, without needing to download the images. We'll keep everyone updated about this once it's ready.

Reply to this email directly or view it on GitHub: https://github.com/barnabytprowe/great3-public/issues/17#issuecomment-41913640