search

drphilmarshall / SpaceWarps

Science Team Website Development and Analysis
MIT License
12 stars 18 forks source link

Introduce SW False Positive Atlas, as a Product #42

Open drphilmarshall opened 10 years ago

drphilmarshall commented 10 years ago

I guess we'll host the catalog and associated PNGs (plus a nice PDF document of them all?) on a website somewhere. I wonder if the zooniverse would be happy to make us a (for now) unlisted folder at spacewarps.org/products where we can put things like this for people to download easily? Is this something to design and then ask for?

anupreeta27 commented 10 years ago

@drphilmarshall could you please ask chris to send me IDs of new FPs and their x,y pos

drphilmarshall commented 10 years ago

​Done, via issue #39​

On Thu, Jul 31, 2014 at 4:56 PM, anupreeta27 notifications@github.com wrote:

@drphilmarshall https://github.com/drphilmarshall could you please ask chris to send me IDs of new FPs

— Reply to this email directly or view it on GitHub https://github.com/drphilmarshall/SpaceWarps/issues/42#issuecomment-50834041 .

anupreeta27 commented 10 years ago

@cpadavis it would be great to have a catalog with IDs and x,y of the FPs. Thanks!

aprajita commented 10 years ago

As part of my task to inspect all 3000 candidates I am going through finding FPs and putting them into categories using Anu's visapp tool. After doing first 1000, Anu & I discussed and to greatly speed up the process she will try to update the visapp with FP categories so that i can just click rather than comment. I need to send Anu these categories (based on our earlier work and what I saw in the top 1000).

drphilmarshall commented 10 years ago

This is great, thanks @aprajita - looks like we forgot to reassign this issue to you back in September, so I just did so. When you have finished the inspection task, we'll need to extract small cutout images of the identified features for the atlas (aka "standard dataset for training ML methods") - should we pass this back to @cpadavis, since he's set up to do that quickly?

aprajita commented 10 years ago

yes definitely, that would be ideal. we can then get cutouts organised by type for the FP atlas we still need to think about where we host this and how we present it, and did we ask zooniverse for spacewarps.org/products ?

anupreeta27 commented 10 years ago

@drphilmarshall I suppose that some of the FPs may be close to the edges. so, it would be better if @aprajita sends me the catalog and i can generate the FP cutouts all of uniform sizes centered on FPs. fine?

aprajita commented 10 years ago

Sure, but It would still be useful to see @cpadavis plots that show marker position density as there may be multiple/different FPs in a single subject

anupreeta27 commented 10 years ago

yes no probs, forgot about it. i meant when the "final" FP catalog is ready, please send it to me to generate the final set of images.

drphilmarshall commented 8 years ago

Re-starting this issue, after @cpadavis assembled the ML training set for the Swinburne group. I got a request from Andy Connolly at UW for a set of gravitational lens images, for ML classification investigation, and I expect others will be forthcoming. I think we should make a simple webpage, linked from our github.io home page, that introduces the image set that Chris made, and provides a link to the tarball for download. If this sounds OK to you, I'll make it and close this issue. I guess we might want to advertize it from the main app as well - and certainly the README of this repo. Thoughts?

anupreeta27 commented 8 years ago

Ok. I don't know what exactly @cpadavis provides but the images+catalog of all SW identified known and new candidates are here - https://github.com/drphilmarshall/SpaceWarps/tree/master/analysis/projects/CFHTLS/candidates

And, I see that the above is not linked from the main app. Should we link this as well?

aprajita commented 8 years ago

seems reasonable to link both files and provide the info on a linked web page.

I might recall incorrectly, but is Chris' dataset based on user click clouds, so is dominated by FPs (with some real lenses) but not duds - or does he also include inspected red galaxies/groups without any features or in his training set that would also help ML training sets. I thnk we had a conversation when I visited SLAC about using every galaxy in the images Chris was wusing to add these field galaxies as duds for ML training set - was this done?

On 13 Apr 2016, at 05:01, anupreeta27 notifications@github.com wrote:

Ok. I don't know what exactly @cpadavis provides but the images+catalog of all SW identified known and new candidates are here - https://github.com/drphilmarshall/SpaceWarps/tree/master/analysis/projects/CFHTLS/candidates

And, I see that the above is not linked from the main app. Should we link this as well?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

drphilmarshall commented 8 years ago

No! It was not - at the moment, we just have features clicked by volunteers, including a mixture of expert-classified lenses and expert-rejected false positives. I guess some of the latter will be LRGs without lensing features but it would be good if we included more of these, I agree. I'd suggest that we post the set that @cpadavis made (and sent to Karl and Colin), and then we iterate later. Good call linking to the candidates as well. I think we should follow Chris' slides and include the example figures from the paper, too - I'll do this. Thanks!

On Wed, Apr 13, 2016 at 5:44 AM, aprajita notifications@github.com wrote:

seems reasonable to link both files and provide the info on a linked web page.

I might recall incorrectly, but is Chris' dataset based on user click clouds, so is dominated by FPs (with some real lenses) but not duds - or does he also include inspected red galaxies/groups without any features or in his training set that would also help ML training sets. I thnk we had a conversation when I visited SLAC about using every galaxy in the images Chris was wusing to add these field galaxies as duds for ML training set - was this done?

On 13 Apr 2016, at 05:01, anupreeta27 notifications@github.com wrote:

Ok. I don't know what exactly @cpadavis provides but the images+catalog of all SW identified known and new candidates are here -

https://github.com/drphilmarshall/SpaceWarps/tree/master/analysis/projects/CFHTLS/candidates

And, I see that the above is not linked from the main app. Should we link this as well?

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub https://github.com/drphilmarshall/SpaceWarps/issues/42#issuecomment-209411694