Remaining subjects are too uncertain to be rejected?

[drphilmarshall]

The SWAP retirement rate has been very low - a few tens per day - for several days now. I think maybe we have reached the point where the remaining active subjects are just bouncing around at intermediate probability, and may continue to do so until they hit their max_classifications limit (or a high skill classifier comes across them). I will investigate this.

If I'm right, we might consider resetting the max_classifications to 20 or something, and finishing sooner rather than later rather than eking out a few more detections/retirements for the next weeks. What do you think?

In any case, there are bound to be a bunch of subjects that finish with probabilities higher than the retirement threshold but lower than the detection threshold. Should we put all of these into Stage 2 along with the Stage 1 detections? (That would make about 7000 subjects, I think, looking at the most recent report). Or should we reject them all? While investigating, Ill see what would happen to the false negative rate in the latter case - at the moment, we know that the false negative rate is 6.6% for the case where we keep all subjects with p > retirement.

[aprajita]

it doesnt make sense to reject them all. these are likely to contain some hard to spot lenses where people disagree. or similarly confusing FPs. Just rejecting them isn't a good option. Perhaps we can put them in as stage 2b (i.e. after the formal stag 2) to keep the main stage 2 sample as the highest probability candidates.

On 23 Nov 2013, at 03:26, Phil Marshall wrote:

The SWAP retirement rate has been very low - a few tens per day - for several days now. I think maybe we have reached the point where the remaining active subjects are just bouncing around at intermediate probability, and may continue to do so until they hit their max_classifications limit (or a high skill classifier comes across them). I will investigate this.

If I'm right, we might consider resetting the max_classifications to 20 or something, and finishing sooner rather than later rather than eking out a few more detections/retirements for the next weeks. What do you think?

In any case, there are bound to be a bunch of subjects that finish with probabilities higher than the retirement threshold but lower than the detection threshold. Should we put all of these into Stage 2 along with the Stage 1 detections? (That would make about 7000 subjects, I think, looking at the most recent report). Or should we reject them all? While investigating, Ill see what would happen to the false negative rate in the latter case - at the moment, we know that the false negative rate is 6.6% for the case where we keep all subjects with p > retirement.

— Reply to this email directly or view it on GitHub.

[anupreeta27]

I think we can include these candidates in stage 2 for now and keep a note of their IDs. At the end of stage 2, the candidates forming the final sample would certainly be those which were detected both in stage 1 & 2 whereas the undecided ones passing the stage 2 can be dealt with separately, if need be.

[anupreeta27]

btw, its kind of sad/worrying that the undecided ones are of the same order as the detected candidates. does it mean we are failing somewhere?

[drphilmarshall]

No, I don't think so - I think we just got focused on the black and white issue of lens or not, retirement or candidate, and forgot about the grey area in between. It's actually kind of interesting that there are not more undecideds - theres a lot of log probability between 2e-7 and 0.95!

I think it will be fine to put them all into Stage 2: if we keep up the refinement message, hopefully most of them should get rejected away. In any case, we'll then have two probabilities to rank by, from stage 1 and stage 2 - I guess these could be labelled Pr(Lens|Optimism) and Pr(Lens|Realism)?

I'm wondering about the max_classifications for stage 2, though. We need some way of ensuring that all teh subjects get a decent number of classifications, say 10-20. One way to enforce this would be to retire after a fixed number, and then just rank by probability at the end. With 7000 subjects and a limit of 50 classifications, we'd be asking for 350,000 classifications, which is 10 days at 35 k per day (which I think we could reach with an email). If we don't set a limit, then the random nature of the subject display will mean that there will be a tail that don't get enough classifications. What do you think?

On Sat, Nov 23, 2013 at 2:32 AM, anupreeta27 notifications@github.comwrote:

btw, its kind of sad/worrying that the undecided ones are of the same order as the detected candidates. does it mean we are failing somewhere?

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29129918 .

[anupreeta27]

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

[drphilmarshall]

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 notifications@github.comwrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491 .

[aprajita]

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 notifications@github.comwrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491 .

— Reply to this email directly or view it on GitHub.

[surhudm]

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 notifications@github.comwrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557 .

[aprajita]

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 notifications@github.comwrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557 .

— Reply to this email directly or view it on GitHub.

[surhudm]

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086 .

[drphilmarshall]

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906 .

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219040367.png ASW0006uy8 0.0000017 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e230.png ASW0007how 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219055550.png ASW0007dev 0.9787273 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219053fa7.png ASW000763d 0.9965136 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219051a99.png ASW0005zlv 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219044383.png ASW00078we 0.9741263 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190528ce.png ASW0005hru 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e93a.png ASW0003odb 0.0059049 43 http://spacewarps.org/subjects/standard/5183f151e4bb210219029e1f.png ASW00037nb 0.9987172 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024977.png ASW00033ru 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235da.png ASW0002fif 0.0132702 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bb07.png ASW0004a4v 0.9862220 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219030c4f.png ASW0004chr 0.9923739 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021903183f.png ASW0005n32 0.9851145 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904041e.png ASW00038jt 0.4348259 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219024e09.png ASW0000vrg 0.0031842 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0cc.png ASW0000vrd 0.1476348 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0c9.png ASW0003apy 0.9874097 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025906.png ASW0002rin 0.9801556 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f7cf.png ASW0006nav 0.9896440 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bb77.png ASW0007a4w 0.0000089 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219052f10.png ASW0000e8n 0.9599184 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219004817.png ASW0002nsh 0.9961025 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e4f1.png ASW000577x 0.9640525 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b3cd.png ASW0000ekq 0.1927546 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190049ca.png ASW00039ga 0.9670172 39 http://spacewarps.org/subjects/standard/5183f151e4bb21021902529a.png ASW00089rh 0.0000151 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905e36d.png ASW00017r5 0.9567529 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dd81.png ASW0005sdf 0.9720690 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219041ee3.png ASW000305c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219022380.png ASW0002mn9 0.9726432 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021901df25.png ASW0007mah 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219056c99.png ASW0007s0z 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190589a3.png ASW0009cs6 0.7532789 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a8f6.png ASW0007l1c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056640.png ASW0006o4y 0.0212719 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bfb2.png ASW0000tc7 0.0003457 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219009487.png ASW0008zlk 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219066638.png ASW0001b39 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ee65.png ASW0006ss1 0.0000017 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d731.png ASW0004zrv 0.0000031 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219038e1b.png ASW0001msw 0.0000012 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190129b0.png ASW0000dok 0.9827726 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219004544.png ASW00073p2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219050e76.png ASW0005wli 0.9882323 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219043446.png ASW0001fpn 0.9526854 28 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105cb.png ASW00040ds 0.9591255 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021902daf0.png ASW0006cgb 0.0000001 6

[anupreeta27]

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall notifications@github.comwrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219040367.png ASW0006uy8 0.0000017 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e230.png ASW0007how 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219055550.png ASW0007dev 0.9787273 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219053fa7.png ASW000763d 0.9965136 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219051a99.png ASW0005zlv 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219044383.png ASW00078we 0.9741263 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190528ce.png ASW0005hru 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e93a.png ASW0003odb 0.0059049 43 http://spacewarps.org/subjects/standard/5183f151e4bb210219029e1f.png ASW00037nb 0.9987172 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024977.png ASW00033ru 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235da.png ASW0002fif 0.0132702 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bb07.png ASW0004a4v 0.9862220 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219030c4f.png ASW0004chr 0.9923739 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021903183f.png ASW0005n32 0.9851145 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904041e.png ASW00038jt 0.4348259 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219024e09.png ASW0000vrg 0.0031842 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0cc.png ASW0000vrd 0.1476348 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0c9.png ASW0003apy 0.9874097 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025906.png ASW0002rin 0.9801556 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f7cf.png ASW0006nav 0.9896440 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bb77.png ASW0007a4w 0.0000089 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219052f10.png ASW0000e8n 0.9599184 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219004817.png ASW0002nsh 0.9961025 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e4f1.png ASW000577x 0.9640525 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b3cd.png ASW0000ekq 0.1927546 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190049ca.png ASW00039ga 0.9670172 39 http://spacewarps.org/subjects/standard/5183f151e4bb21021902529a.png ASW00089rh 0.0000151 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905e36d.png ASW00017r5 0.9567529 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dd81.png ASW0005sdf 0.9720690 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219041ee3.png ASW000305c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219022380.png ASW0002mn9 0.9726432 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021901df25.png ASW0007mah 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219056c99.png ASW0007s0z 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190589a3.png ASW0009cs6 0.7532789 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a8f6.png ASW0007l1c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056640.png ASW0006o4y 0.0212719 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bfb2.png ASW0000tc7 0.0003457 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219009487.png ASW0008zlk 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219066638.png ASW0001b39 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ee65.png ASW0006ss1 0.0000017 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d731.png ASW0004zrv 0.0000031 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219038e1b.png ASW0001msw 0.0000012 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190129b0.png ASW0000dok 0.9827726 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219004544.png ASW00073p2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219050e76.png ASW0005wli 0.9882323 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219043446.png ASW0001fpn 0.9526854 28 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105cb.png ASW00040ds 0.9591255 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021902daf0.png ASW0006cgb 0.0000001 6

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29599833 .

CFHTLS_001_1554 30.5

[aprajita]

Hi all,

thanks for the explanations

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

hmm, i've got very lost with the timeline, does stage 2 really need to be completed in 2 weeks - can it not continue while we work on reconfiguring the site for VICS82?

i think i agree doing stage 2 with P>0.95 is a preferred option, as this is our key sample. but perhaps we could do stage 2e-7<P<0.95 after VICS82?

i agree, we do need to understand where the real systems are. when we did this early in the project an large number were in the rejected or undecided piles. so this is a vital step. i was quite concerned about our earlier status with the real systems, hopefully as the project has progressed and the users are more experienced, the missed real/known lenses are few. but if we are still incomplete we need to learn why before the next project (either for VICS82, or the next dataset), and see if we can incorporate what we've learned into the tutorials, sims or other project design. If many real/knowns are still undecided, this argues for doing stage 2B with the undecided objects but as I said before I think this can happen post VICS82.

thanks aprajita.

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita notifications@github.com wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906 .

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219040367.png ASW0006uy8 0.0000017 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e230.png ASW0007how 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219055550.png ASW0007dev 0.9787273 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219053fa7.png ASW000763d 0.9965136 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219051a99.png ASW0005zlv 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219044383.png ASW00078we 0.9741263 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190528ce.png ASW0005hru 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e93a.png ASW0003odb 0.0059049 43 http://spacewarps.org/subjects/standard/5183f151e4bb210219029e1f.png ASW00037nb 0.9987172 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024977.png ASW00033ru 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235da.png ASW0002fif 0.0132702 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bb07.png ASW0004a4v 0.9862220 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219030c4f.png ASW0004chr 0.9923739 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021903183f.png ASW0005n32 0.9851145 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904041e.png ASW00038jt 0.4348259 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219024e09.png ASW0000vrg 0.0031842 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0cc.png ASW0000vrd 0.1476348 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0c9.png ASW0003apy 0.9874097 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025906.png ASW0002rin 0.9801556 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f7cf.png ASW0006nav 0.9896440 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bb77.png ASW0007a4w 0.0000089 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219052f10.png ASW0000e8n 0.9599184 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219004817.png ASW0002nsh 0.9961025 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e4f1.png ASW000577x 0.9640525 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b3cd.png ASW0000ekq 0.1927546 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190049ca.png ASW00039ga 0.9670172 39 http://spacewarps.org/subjects/standard/5183f151e4bb21021902529a.png ASW00089rh 0.0000151 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905e36d.png ASW00017r5 0.9567529 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dd81.png ASW0005sdf 0.9720690 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219041ee3.png ASW000305c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219022380.png ASW0002mn9 0.9726432 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021901df25.png ASW0007mah 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219056c99.png ASW0007s0z 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190589a3.png ASW0009cs6 0.7532789 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a8f6.png ASW0007l1c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056640.png ASW0006o4y 0.0212719 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bfb2.png ASW0000tc7 0.0003457 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219009487.png ASW0008zlk 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219066638.png ASW0001b39 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ee65.png ASW0006ss1 0.0000017 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d731.png ASW0004zrv 0.0000031 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219038e1b.png ASW0001msw 0.0000012 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190129b0.png ASW0000dok 0.9827726 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219004544.png ASW00073p2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219050e76.png ASW0005wli 0.9882323 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219043446.png ASW0001fpn 0.9526854 28 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105cb.png ASW00040ds 0.9591255 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021902daf0.png ASW0006cgb 0.0000001 6 — Reply to this email directly or view it on GitHub.

[drphilmarshall]

That's great, thanks Anu! I'll try and take a look today.

On Mon, Dec 2, 2013 at 7:11 AM, anupreeta27 notifications@github.comwrote:

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall notifications@github.comwrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com>

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219040367.png ASW0006uy8 0.0000017 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e230.png ASW0007how 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219055550.png ASW0007dev 0.9787273 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219053fa7.png ASW000763d 0.9965136 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219051a99.png ASW0005zlv 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219044383.png ASW00078we 0.9741263 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190528ce.png ASW0005hru 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e93a.png ASW0003odb 0.0059049 43 http://spacewarps.org/subjects/standard/5183f151e4bb210219029e1f.png ASW00037nb 0.9987172 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024977.png ASW00033ru 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235da.png ASW0002fif 0.0132702 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bb07.png ASW0004a4v 0.9862220 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219030c4f.png ASW0004chr 0.9923739 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021903183f.png ASW0005n32 0.9851145 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904041e.png ASW00038jt 0.4348259 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219024e09.png ASW0000vrg 0.0031842 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0cc.png ASW0000vrd 0.1476348 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0c9.png ASW0003apy 0.9874097 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025906.png ASW0002rin 0.9801556 6 http://spacewarps.org/subjects/standard/5183f151e4bb21

[drphilmarshall]

Good ideas: let's ask Laura about leaving stage 2 running over Christmas - I can't think why we wouldn't be able to do that! @laurawhyte, @chrissnyder, would this be OK?

Probs better to do CFTLS till its done and then move on without coming back, but at a pinch we could come back, yes.

On Mon, Dec 2, 2013 at 7:20 AM, aprajita notifications@github.com wrote:

Hi all,

thanks for the explanations

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

hmm, i've got very lost with the timeline, does stage 2 really need to be completed in 2 weeks - can it not continue while we work on reconfiguring the site for VICS82?

i think i agree doing stage 2 with P>0.95 is a preferred option, as this is our key sample. but perhaps we could do stage 2e-7<P<0.95 after VICS82?

i agree, we do need to understand where the real systems are. when we did this early in the project an large number were in the rejected or undecided piles. so this is a vital step. i was quite concerned about our earlier status with the real systems, hopefully as the project has progressed and the users are more experienced, the missed real/known lenses are few. but if we are still incomplete we need to learn why before the next project (either for VICS82, or the next dataset), and see if we can incorporate what we've learned into the tutorials, sims or other project design. If many real/knowns are still undecided, this argues for doing stage 2B with the undecided objects but as I said before I think this can happen post VICS82.

thanks aprajita.

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com> wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219040367.png ASW0006uy8 0.0000017 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e230.png ASW0007how 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219055550.png ASW0007dev 0.9787273 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219053fa7.png ASW000763d 0.9965136 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219051a99.png ASW0005zlv 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219044383.png ASW00078we 0.9741263 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190528ce.png ASW0005hru 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e93a.png ASW0003odb 0.0059049 43 http://spacewarps.org/subjects/standard/5183f151e4bb210219029e1f.png ASW00037nb 0.9987172 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024977.png ASW00033ru 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235da.png ASW0002fif 0.0132702 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bb07.png ASW0004a4v 0.9862220 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219030c4f.png ASW0004chr 0.9923739 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021903183f.png ASW0005n32 0.9851145 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904041e.png ASW00038jt 0.4348259 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219024e09.png ASW0000vrg 0.0031842 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0cc.png ASW0000vrd 0.1476348 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a0c9.png ASW0003apy 0.9874097 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025906.png ASW0002rin 0.9801556 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f7cf.png ASW0006nav 0.9896440 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bb77.png ASW0007a4w 0.0000089 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219052f10.png ASW0000e8n 0.9599184 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219004817.png ASW0002nsh 0.9961025 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e4f1.png ASW000577x 0.9640525 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b3cd.png ASW0000ekq 0.1927546 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190049ca.png ASW00039ga 0.9670172 39 http://spacewarps.org/subjects/standard/5183f151e4bb21021902529a.png ASW00089rh 0.0000151 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905e36d.png ASW00017r5 0.9567529 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dd81.png ASW0005sdf 0.9720690 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219041ee3.png ASW000305c 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219022380.png ASW0002mn9 0.9726432 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021901df25.png ASW0007mah 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219056c99.png ASW0007s0z 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190589a3.png ASW0009cs6 0.7532789 17 http://space

[drphilmarshall]

Hi Anu,

My lens catalog, finknwnlens_sg_stg2_catalog.txt, seems not to contain the search strings I need!:

Problem - name in $lenscat does not match name in manifest! Eg:

First line of $lenscat:

CFHTLS_001_12102_1549_o_gri.png GROUP A

Look for it in listfiles_d1_d11

grep CFHTLS_001_12102 $manifest

No result! Email anu...

Looks like the naming system is different between the two files, somehow...

Phil

On Mon, Dec 2, 2013 at 3:28 PM, Phil Marshall dr.phil.marshall@gmail.comwrote:

That's great, thanks Anu! I'll try and take a look today.

On Mon, Dec 2, 2013 at 7:11 AM, anupreeta27 notifications@github.comwrote:

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall notifications@github.comwrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com>

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219001ebc.png ASW0002lcj 0.9786308 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d893.png ASW00076ft 0.0000405 35 http://spacewarps.org/subjects/standard/5183f151e4bb210219051c59.png ASW0002ooc 0.9983003 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e96c.png ASW0000hng 0.9851422 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021900595c.png ASW0000hn5 0.7663930 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219005951.png ASW0002d5f 0.9970523 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901af13.png ASW0004oy7 0.9924718 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021903574f.png ASW0004oy1 0.0001094 44 http://spacewarps.org/subjects/standard/5183f151e4bb210219035749.png ASW0004oxy 0.9904959 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219035746.png ASW0007mvl 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219056f91.png ASW0008zre 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906670a.png ASW0002a8e 0.9934290 46 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a04e.png ASW0003vs4 0.0001027 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021902c3a4.png ASW0002ixt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cc61.png ASW0006rp2 0.0017661 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d1b6.png ASW0004lbf 0.9971705 27 http://spacewarps.org/subjects/standard/5183f151e4bb2102190344eb.png ASW00024fs 0.9857095 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182f8.png ASW000523i 0.9724684 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190399de.png ASW0006ndp 0.9615702 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bbdd.png ASW0000pk9 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219008169.png ASW0005mxz 0.9561666 11 http://spacewarps.org/

[drphilmarshall]

OK, thanks Anu (she sent me an updated catalog offline).

We have 204 known lenses; 105 got retired, leaving 99 of them still active. Of these, 94 have P > 0.95! This means that the completeness to known lenses is 46% at P > 0.95, and 49% at P > 2e-7.

Good and bad news! Overall we are not as complete as we would have liked; but, there is little to be gained by inspecting more than the P > 0.95 sample of candidates at Stage 2. Shall we proceed with max_classifications = 40 with Stage 2 restructed to candidates with P > 0.95?

(When Stage 1 is complete, that is - still waiting to hear from @parrish about the impact of setting max_classifications = 20 to finish of stage 1)

On Mon, Dec 2, 2013 at 4:08 PM, Phil Marshall dr.phil.marshall@gmail.comwrote:

Hi Anu,

My lens catalog, finknwnlens_sg_stg2_catalog.txt, seems not to contain the search strings I need!:

Problem - name in $lenscat does not match name in manifest! Eg:

First line of $lenscat:

CFHTLS_001_12102_1549_o_gri.png GROUP A

Look for it in listfiles_d1_d11

grep CFHTLS_001_12102 $manifest

No result! Email anu...

Looks like the naming system is different between the two files, somehow...

Phil

On Mon, Dec 2, 2013 at 3:28 PM, Phil Marshall dr.phil.marshall@gmail.comwrote:

That's great, thanks Anu! I'll try and take a look today.

On Mon, Dec 2, 2013 at 7:11 AM, anupreeta27 notifications@github.comwrote:

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall notifications@github.comwrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com

wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com>

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219057ffe.png ASW0008fao 0.9929790 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ff70.png ASW0008wmr 0.9617338 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065733.png ASW00024v4 0.9750636 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018520.png ASW00015gx 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d1f1.png ASW0008xnm 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c62.png ASW0008xnt 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219065c69.png ASW00024o5 0.9916879 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219018425.png ASW00034y8 0.9985979 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219023bd0.png ASW0004ilg 0.9885249 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219033724.png ASW0003ldo 0.9775215 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219028efc.png ASW0004ho2 0.9863236 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219033272.png ASW0005vby 0.0696339 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219042dde.png ASW0004c48 0.9648996 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219031658.png ASW00094m6 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219067f9e.png ASW00071q9 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219050481.png ASW0000qm0 0.9510121 25 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086b8.png ASW00049x3 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219030b37.png ASW00002c4 0.9753796 36 http://spacewarps.org/subjects/standard/5183f151e4bb210219000bd4.png ASW0003y0i 0.9536635 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cef2.png ASW0008j7j 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906133f.png ASW0003y0u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902cefe.png ASW0008j78 0.0000483 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219061334.png ASW0000ths 0.0011051 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219009550.png ASW0006cle 0.0001055 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219048542.png ASW0007qw3 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190583e3.png ASW00025s0 0.0000101 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190189c0.png ASW0004x9c 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219038160.png ASW0002s9l 0.9947038 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fb99.png ASW0001342 0.9647202 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c602.png ASW0004pas 0.9982249 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219035914.png ASW0006bo3 0.9919719 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219048093.png ASW0001d8z 0.9917411 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f953.png ASW00011d8 0.9918883 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021900bd2c.png ASW0009cg6 0.9715719 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a746.png ASW0000pfp 0.9542152 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190080c5.png ASW0002fhg 0.9829774 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901bae4.png ASW0002tkm 0.0062730 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020236.png ASW00053oz 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1f3.png ASW00053pa 0.9885895 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a1fe.png ASW00001jn 0.9894834 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d3.png ASW00001jl 0.9963137 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190007d1.png ASW00021th 0.0024743 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190175b5.png ASW0006xcu 0.0001832 38 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ee5e.png ASW000062k 0.9754784 38

[aprajita]

oh dear, I was hoping the earlier trends would have been overcome by expereince. A quick check to see if experience plays a role would be to see if the number of real lenses passing P>0.95 increases with dataset number.

I agree we only really need to view P>0.95 for stage 2

we should look carefully at the images and classification histories of the rejected known lenses in case this is an issue we could have addressed in swap (e.g. user weighting) or if these are just tricky to spot systems.

if you have chance, could you please make a gallery of those known/real lenses with P>0.95, those rejected, and the handful in between.

thanks aprajita.

On 3 Dec 2013, at 01:42, Phil Marshall wrote:

OK, thanks Anu (she sent me an updated catalog offline).

We have 204 known lenses; 105 got retired, leaving 99 of them still active. Of these, 94 have P > 0.95! This means that the completeness to known lenses is 46% at P > 0.95, and 49% at P > 2e-7.

Good and bad news! Overall we are not as complete as we would have liked; but, there is little to be gained by inspecting more than the P > 0.95 sample of candidates at Stage 2. Shall we proceed with max_classifications = 40 with Stage 2 restructed to candidates with P > 0.95?

(When Stage 1 is complete, that is - still waiting to hear from @parrish about the impact of setting max_classifications = 20 to finish of stage 1)

On Mon, Dec 2, 2013 at 4:08 PM, Phil Marshall dr.phil.marshall@gmail.comwrote:

Hi Anu,

My lens catalog, finknwnlens_sg_stg2_catalog.txt, seems not to contain the search strings I need!:

Problem - name in $lenscat does not match name in manifest! Eg:

First line of $lenscat:

CFHTLS_001_12102_1549_o_gri.png GROUP A

Look for it in listfiles_d1_d11

grep CFHTLS_001_12102 $manifest

No result! Email anu...

Looks like the naming system is different between the two files, somehow...

Phil

On Mon, Dec 2, 2013 at 3:28 PM, Phil Marshall dr.phil.marshall@gmail.comwrote:

That's great, thanks Anu! I'll try and take a look today.

On Mon, Dec 2, 2013 at 7:11 AM, anupreeta27 notifications@github.comwrote:

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall notifications@github.comwrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P > 0.95, that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita notifications@github.com

wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com>

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38 http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1 http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33 http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39 http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22 http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34 http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43 http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15 http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48 http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12 http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40 http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36 http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48 http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1 http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11 http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20 http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31 http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46 http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12 http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29 http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24 http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13 http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19 http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3 http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25 http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14 http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26 http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30 http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39 http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11 http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49 http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71 http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41 http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27 http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9 http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14 http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32 http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37 http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28 http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31 http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37 http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25 http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7 http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18 http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45 http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23 http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15 http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42 http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43 http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17 http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24 http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22 http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10 http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35 http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33 http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3 http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42 http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4 http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51 http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29 http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27 http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9 http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17 http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb210219054151.png ASW0002pcj 0.0000001 8 http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecd3.png ASW00051wh 0.0009285 36 http://spacewarps.org/subjects/standard/5183f151e4bb2102190398e1.png ASW0007b76 0.9587350 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219053472.png ASW0004cmh 0.9556610 4 http://spacewarps.org/subjects/standard/5183f151e4bb2102190318e9.png ASW0007y3m 0.0000001 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021905a862.png ASW0000z1f 0.9930174 15 http://spacewarps.org/subjects/standard/5183f151e4bb21021900b163.png ASW0004o9l 0.0000001 21 http://spacewarps.org/subjects/standard/5183f151e4bb2102190353d9.png ASW0006kf9 0.0000001 5 http://spacewarps.org/subjects/standard/5183f151e4bb21021904ace5.png ASW0002dk3 0.0000003 50 http://spacewarps.org/subjects/standard/5183f151e4bb21021901b123.png ASW0002rfa 0.9783538 34 http://spacewarps.org/subjects/standard/5183f151e4bb21021901f756.png ASW0004kmv 0.0006433 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219034177.png ASW00025qd 0.9743097 33 http://spacewarps.org/subjects/standard/5183f151e4bb210219018985.png ASW00025ql 0.9944987 24 http://spacewarps.org/subjects/standard/5183f151e4bb21021901898d.png ASW000129x 0.9930102 12 http://spacewarps.org/subjects/standard/5183f151e4bb21021900c1c5.png ASW0006ksp 0.9707300 4 http://spacewarps.org/subjects/standard/5183f151e4bb21021904aec9.png ASW0000d02 0.0001538 51 http://spacewarps.org/subjects/standard/5183f151e4bb2102190041d2.png ASW0002s24 0.9833874 7 http://spacewarps.org/subjects/standard/5183f151e4bb21021901fa8c.png ASW0005yv1 0.9938474 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219043fbd.png ASW0003c6w 0.9831795 30 http://spacewarps.org/subjects/standard/5183f151e4bb210219026078.png ASW0009cis 0.0000002 22 http://spacewarps.org/subjects/standard/5183f151e4bb21021906a7a4.png ASW0001ols 0.9763383 11 http://spacewarps.org/subjects/standard/5183f151e4bb2102190132d0.png ASW0009651 0.0000001 13 http://spacewarps.org/subjects/standard/5183f151e4bb210219068755.png ASW00030it 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219022565.png ASW0007l4j 0.0000024 20 http://spacewarps.org/subjects/standard/5183f151e4bb2102190566b3.png ASW0000g5i 0.9816908 34 http://spacewarps.org/subjects/standard/5183f151e4bb2102190051c6.png ASW0006hoa 0.9863088 6 http://spacewarps.org/subjects/standard/5183f151e4bb210219049efa.png ASW0003e8m 0.0448873 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219026ad6.png ASW0000l2b 0.0002404 50 http://spacewarps.org/subjects/standard/5183f151e4bb210219006aa3.png ASW0001ydq 0.9984618 20 http://spacewarps.org/subjects/standard/5183f151e4bb21021901644e.png ASW0003edh 0.0000001 10 http://spacewarps.org/subjects/standard/5183f151e4bb210219026b85.png ASW0007kjh 0.9790386 8 http://spacewarps.org/subjects/standard/5183f151e4bb2102190563bd.png ASW0003bit 0.9664208 16 http://spacewarps.org/subjects/standard/5183f151e4bb210219025d15.png ASW0007q4e 0.0000001 6 http://spacewarps.org/subjects/standard/5183f151e4bb21021905

[drphilmarshall]

OK, I think we are converging. P > 0.95 subjects, 40 classifications each.

In the web directory below are (or will be) galleries of detected (P > 0.95), rejected (P < 2e-7) and undecided subjects (as of last Thursday). Looks to me like the rejections are either genuinely hard OR they are lenses that contain sims! These should have been caught and reinjected (in D11 was it?). Were the sim subjects not put back in properly, with simulated arcs removed?

http://www.slac.stanford.edu/~pjm/SpaceWarps/Science/analysis/knownlenses/

@parrish, ASW0001e2j is a test subject that seems to contain a sim. Can you investigate this please? Which dataset is it? Do its metadata say sim or test subject? If it is indeed a test subject, why do I see it with a sim in it? Or conversely, since it is a sim, why does SWAP think it is a test subject? Thanks! (Here it is in Talk: http://talk.spacewarps.org/#/subjects/ASW0001e2j)

On Mon, Dec 2, 2013 at 6:07 PM, aprajita notifications@github.com wrote:

oh dear, I was hoping the earlier trends would have been overcome by expereince. A quick check to see if experience plays a role would be to see if the number of real lenses passing P>0.95 increases with dataset number.

I agree we only really need to view P>0.95 for stage 2

we should look carefully at the images and classification histories of the rejected known lenses in case this is an issue we could have addressed in swap (e.g. user weighting) or if these are just tricky to spot systems.

if you have chance, could you please make a gallery of those known/real lenses with P>0.95, those rejected, and the handful in between.

thanks aprajita.

On 3 Dec 2013, at 01:42, Phil Marshall wrote:

OK, thanks Anu (she sent me an updated catalog offline).

We have 204 known lenses; 105 got retired, leaving 99 of them still active. Of these, 94 have P > 0.95! This means that the completeness to known lenses is 46% at P > 0.95, and 49% at P > 2e-7.

Good and bad news! Overall we are not as complete as we would have liked; but, there is little to be gained by inspecting more than the P > 0.95 sample of candidates at Stage 2. Shall we proceed with max_classifications = 40 with Stage 2 restructed to candidates with P > 0.95?

(When Stage 1 is complete, that is - still waiting to hear from @parrish about the impact of setting max_classifications = 20 to finish of stage 1)

On Mon, Dec 2, 2013 at 4:08 PM, Phil Marshall < dr.phil.marshall@gmail.com>wrote:

Hi Anu,

My lens catalog, finknwnlens_sg_stg2_catalog.txt, seems not to contain the search strings I need!:

Problem - name in $lenscat does not match name in manifest! Eg:

First line of $lenscat:

CFHTLS_001_12102_1549_o_gri.png GROUP A

Look for it in listfiles_d1_d11

grep CFHTLS_001_12102 $manifest

No result! Email anu...

Looks like the naming system is different between the two files, somehow...

Phil

On Mon, Dec 2, 2013 at 3:28 PM, Phil Marshall < dr.phil.marshall@gmail.com>wrote:

That's great, thanks Anu! I'll try and take a look today.

On Mon, Dec 2, 2013 at 7:11 AM, anupreeta27 notifications@github.comwrote:

hi phil, i will get to it in a day or two but if i don't here's the known lens catalog with cfht id and you can match it to the catalog here to get the corresp. asw_id http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11 cheers, Anu

On Mon, Dec 2, 2013 at 4:53 PM, Phil Marshall < notifications@github.com>wrote:

Hi,

I checked the code and we are not retiring test subjects as they cross the 0.95 barrier, but we are ignoring classifications made of these still-active subjects once they cross the detection threshold. (We do the same thing with the rejections - as soon as we see them cross the threshold we flag them for retirement, and then SWITCH them when SWAP has finished running. But there is always some time between the threshold being crossed and SWAP seeing that happen - so most subjects get more classifications than we use). So yes, it's as if we are retiring subjects once they cross the 0.95 line, which them makes the numbers above make sense.

I think the subjects with 0.999 are there by chance - they cross the threshold and overshoot. The Stage 1 probability is noisy - which is why we need Stage 2.

The completeness is only an approximation because 1) its based on sims not real lenses and 2) the sims get a lot more classifications than test subjects. We'd get a more accurate estimate by somehow treating the sims and duds in the same way as the test subjects, and only counting the classifications up till they reached the threshold. I should do this for the final analysis - but it wouldn't affect the probabilities of the test subjects we have now.

I guess the max_classifications for Stage 2 has also to be set by expediency - we only have a short time to do this, before reconfiguring for VICS82. Suppose we give ourselves 2 weeks to do Stage 2; thats probably 150k classifications at this point. With the 3500 candidates at P

0.95,

that would be about 40 classifications per subject. With the 7000 candidates at P > 2e-7, we could only afford 20 classifications per subject. I think my preference would be the former combination - fewer candidates with more classifications each, because we want good data on these systems. What do you think?

One test we could do before we decide this is to ask what the probabilities of the real lenses are. I can't do this though, because I do not have the zoo IDs of the known lenses. Anu, can you match the candidate catalog that is attached to this email to the known lens catalog please? Then we can look at real lens completeness as a function of SWAP probability :-)

Cheers

Phil

On Thu, Nov 28, 2013 at 3:38 AM, surhudm notifications@github.com

wrote:

I was under the impression that we were retiring sims too, but of course that wont make sense, as all of them will get retired pretty soon then given their frequent visibility. So there must be some more subtle reason, still thinking ....

Surhud

On Thu, Nov 28, 2013 at 7:45 PM, aprajita < notifications@github.com>

wrote:

Hi Surhud

I think I got confused between the probabilities and the completeness. Yes, we do retire candidates above p>0.95, but the completeness is determined from simulations that aren't retired.

What I found odd is that the completeness is constant from P0.01 to P0.95. I don't quite understand why completeness defined by the simulations (whose probabilities don't get frozen) means that there is this distinct step at 0.95 which is set for real subjects only, and from what I recall was a rather ad hoc choice. But I suspect I am just not clear about how Phil defines completeness. I see what you mean though, once a real subject reaches 0.95 it gets frozen so we should expect increasing incompleteness above p>0.95 as subjects will not continue to be ranked above 0.95, i.e. whether a subject reaches 0.95 or any value above 0.95 it is one and the same. But this is not true for sims as they can continue to be ranked.

Aprajita.

On 28 Nov 2013, at 10:27, surhudm wrote:

Hi guys,

Sorry for not replying before.

What is the threshold currently? Isn't it 0.95? Then it makes sense that 0.95 appears special. We are retiring subjects after they cross the threshold and then the probabilities get frozen. These candidates do not receive any more classification to further increase their probabilities. And hence the dwindling number of high probability candidates.

Isn't this what is happening or am I missing something?

Cheers, Surhud

On Thu, Nov 28, 2013 at 6:52 PM, aprajita < notifications@github.com>

wrote:

On 28 Nov 2013, at 03:42, Phil Marshall wrote:

OK, I looked into the undecideds a bit - see the attached plot that SWAP now spits out, of the candidates (P > 0.95) and the undecideds (2e-7 < P < 0.95, N < Nmax). The distribution of P values is basically uniform in log P except for a peak at 2e-7 (the undecideds that are just about to get rejected). So, most of the undecideds are indeed just rattling around at random. There are about 3000 of them - and of course most of them will not contain lenses. To quantify this, I looked at the sims that are undecided too, and computed the completeness we would get by defining a sample to have P > threshold, for various threshold values. Here's what I get:

P > 0.01: 3837 candidates at 93.0% completeness P > 0.05: 3675 candidates at 92.9% completeness P > 0.10: 3609 candidates at 92.9% completeness P > 0.25: 3514 candidates at 92.9% completeness P > 0.50: 3456 candidates at 92.8% completeness P > 0.90: 3358 candidates at 92.8% completeness P > 0.95: 3348 candidates at 92.8% completeness

Dialing down the detection threshold from 95% to 1% has almost no impact on our completeness! SW generates a very clean separation between lens and non lens.

OK, but this seems wierd to me. doesn't that just mean that we only have <~7% tricky simulations and so if we are missing hard to spot lenses they are more likely to just be rejected outright than to be floating around in the undecided category. Just becuase our simulations have <~7% tricky subjects doesn't mean the real lenses will have the same.

P = 0.95 seems to be somewhat special - if I look at how the completeness changes above that I get:

P > 0.95: 3348 candidates at 92.8% completeness P > 0.96: 2921 candidates at 82.9% completeness P > 0.97: 2463 candidates at 71.8% completeness P > 0.99: 1153 candidates at 37.6% completeness P > 0.995: 581 candidates at 23.8% completeness P > 0.999: 37 candidates at 2.2% completeness

that is odd, why should 0.95 be so special - hmmm

indicating that the P value is somewhat noisy. I looked at the "top 40" candidates, that have P > 0.999 - only a handful of them look like lenses. So the stage 1 P values are only useful for thresholding at around 0.95.

worrying that only a handful for P>0.999 are lenses can you give the average number of classifications for the P ranges P>0.95 so we can see if this is having any impact before we fix max classifications for stage 2. I worry that the ratings of experts are still promoting candidates to high rankings in few classifications.

I'll ask Michael to set max_classifications to 20, to finish off the stage 1 sample. There are only about 1000 subjects with N < 20, and so these should get cleaned up once the Nmax = 20 retirement threshold kicks in.

i'm not convinced about N~20 and would prefer that everyone sees everything, but i understand that would mean removing the random element of the image server.

thanks aprajita.

I think we could then leave it there for stage 2, as Anu says.

Cheers

Phil

On Sun, Nov 24, 2013 at 8:04 PM, anupreeta27 < notifications@github.com>wrote:

max_classf ~ 10-20 sounds good. hopefully, the new training sample will help us get rid of a substantial fraction of the FPs in stage 2.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29176491>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29451557>

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29455086>

.

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29457906>

.

zooid P Nclass image

ASW0000jct 0.0007977 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190061fd.png ASW0000jch 0.3417965 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190061f1.png ASW0001abu 0.9752075 41

http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8a.png ASW0001aby 0.9943662 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021900ea8e.png ASW0005hf8 0.0001361 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021903e774.png ASW00046q1 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021902fb09.png ASW0005pnc 0.9958981 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219041118.png ASW0002qyo 0.9986747 45

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f500.png ASW0001hsu 0.9941071 11

http://spacewarps.org/subjects/standard/5183f151e4bb21021901105e.png ASW0006oje 0.9974437 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021904c1ba.png ASW0000jzl 0.0000002 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219006531.png ASW0000jjg 0.9754032 47

http://spacewarps.org/subjects/standard/5183f151e4bb2102190062ec.png ASW0006pvd 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021904c879.png ASW0008swn 0.9894848 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219064457.png ASW0007lwn 0.2453415 38

http://spacewarps.org/subjects/standard/5183f151e4bb210219056aa7.png ASW0004q1x 0.9940271 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219035ce5.png ASW000075q 0.0002614 1

http://spacewarps.org/subjects/standard/5183f151e4bb21021900243e.png ASW0006si1 0.0000980 25

http://spacewarps.org/subjects/standard/5183f151e4bb21021904d5c9.png ASW00095rj 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021906856f.png ASW0008yd7 0.0000001 4

http://spacewarps.org/subjects/standard/5183f151e4bb210219065ffb.png ASW00022hg 0.9733038 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219017914.png ASW0002u2b 0.0000001 33

http://spacewarps.org/subjects/standard/5183f151e4bb2102190204b3.png ASW0002dj5 0.9916044 15

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b101.png ASW0000rio 0.1137926 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219008b50.png ASW0000ljz 0.0000001 40

http://spacewarps.org/subjects/standard/5183f151e4bb210219006d1f.png ASW0001tps 0.9800645 41

http://spacewarps.org/subjects/standard/5183f151e4bb210219014cb0.png ASW00057hp 0.0000001 21

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b52d.png ASW0002emy 0.9644195 30

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b69a.png ASW0002emv 0.9930991 49

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b697.png ASW00045rr 0.9752209 49

http://spacewarps.org/subjects/standard/5183f151e4bb21021902f637.png ASW0005m2b 0.9897292 24

http://spacewarps.org/subjects/standard/5183f151e4bb21021903fef3.png ASW0002lr1 0.0004079 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901da9d.png ASW0005rrz 0.9627042 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219041bdf.png ASW0001839 0.0023297 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900df35.png ASW0000um8 0.9805743 39

http://spacewarps.org/subjects/standard/5183f151e4bb210219009b00.png ASW0005sww 0.9982429 10

http://spacewarps.org/subjects/standard/5183f151e4bb2102190421a0.png ASW00062y3 0.9982688 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021904546b.png ASW0008vtn 0.0000896 17

http://spacewarps.org/subjects/standard/5183f151e4bb21021906531b.png ASW00009h3 0.9702103 31

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ff7.png ASW0003902 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219025052.png ASW00009ha 0.0003523 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffe.png ASW00009h7 0.0001277 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffb.png ASW00009h8 0.0000003 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ffc.png ASW0005vf5 0.0000001 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219042e51.png ASW0004end 0.9771851 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219032329.png ASW0001mlw 0.9885023 15

http://spacewarps.org/subjects/standard/5183f151e4bb2102190128b4.png ASW0001dnu 0.9724822 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021900fb6a.png ASW0000982 0.0261153 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002eb2.png ASW00058ln 0.9983172 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021903bacb.png ASW00019hr 0.9969794 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021900e64f.png ASW000564l 0.9683462 30

http://spacewarps.org/subjects/standard/5183f151e4bb21021903ae45.png ASW0001hsg 0.9981789 46

http://spacewarps.org/subjects/standard/5183f151e4bb210219011050.png ASW0001mbl 0.0000041 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219012741.png ASW0001mbz 0.9680874 12

http://spacewarps.org/subjects/standard/5183f151e4bb21021901274f.png ASW00008hh 0.9670128 37

http://spacewarps.org/subjects/standard/5183f151e4bb210219002af5.png ASW0000f8y 0.5298179 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219004d32.png ASW00008hm 0.1669801 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002afa.png ASW00018vu 0.9717728 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021900e33a.png ASW0009a8b 0.9915267 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219069c0b.png ASW0002r6d 0.9617929 17

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f615.png ASW0006ejj 0.9731393 3

http://spacewarps.org/subjects/standard/5183f151e4bb210219048f1f.png ASW0005ebp 0.0000001 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021903d7c5.png ASW0004wuf 0.9991881 30

http://spacewarps.org/subjects/standard/5183f151e4bb210219037f47.png ASW00025y2 0.3597958 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219018a9a.png ASW0005u3l 0.9938785 6

http://spacewarps.org/subjects/standard/5183f151e4bb2102190427a1.png ASW000821u 0.0000038 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021905bc62.png ASW000069x 0.1462216 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219001fc5.png ASW0004ne0 0.9530076 9

http://spacewarps.org/subjects/standard/5183f151e4bb210219034f68.png ASW0009bb3 0.0000018 21

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a17f.png ASW00063kj 0.9788980 13

http://spacewarps.org/subjects/standard/5183f151e4bb210219045793.png ASW0006oqk 0.1539656 31

http://spacewarps.org/subjects/standard/5183f151e4bb21021904c2bc.png ASW00051j4 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219039700.png ASW0006zkf 0.0010774 18

http://spacewarps.org/subjects/standard/5183f151e4bb21021904f98f.png ASW0002ucd 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021902061d.png ASW00006s9 0.0000010 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002259.png ASW00012kq 0.9711313 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021900c34a.png ASW0002cso 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad48.png ASW00009mv 0.8451406 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190030c7.png ASW0008598 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021905cc9c.png ASW00052s9 0.9971435 15

http://spacewarps.org/subjects/standard/5183f151e4bb210219039d59.png ASW0006rj6 0.0000005 25

http://spacewarps.org/subjects/standard/5183f151e4bb21021904d0e2.png ASW0008o83 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219062ca3.png ASW0006rtm 0.0000001 10

http://spacewarps.org/subjects/standard/5183f151e4bb21021904d25a.png ASW0002754 0.9847435 14

http://spacewarps.org/subjects/standard/5183f151e4bb2102190190a8.png ASW0009d7c 0.9873836 15

http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab18.png ASW00022ti 0.9912356 25

http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac6.png ASW00022tj 0.9913521 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219017ac7.png ASW0009d7g 0.0000261 19

http://spacewarps.org/subjects/standard/5183f151e4bb21021906ab1c.png ASW00015ou 0.0000005 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900d30e.png ASW00072fj 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021905080f.png ASW0008r8n 0.9537904 3

http://spacewarps.org/subjects/standard/5183f151e4bb210219063be7.png ASW00023e3 0.9622513 31

http://spacewarps.org/subjects/standard/5183f151e4bb210219017dab.png ASW00097tc 0.9644170 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219068fd0.png ASW0001r0q 0.9992778 45

http://spacewarps.org/subjects/standard/5183f151e4bb210219013f0a.png ASW0002bmu 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a766.png ASW0007dr4 0.0105957 26

http://spacewarps.org/subjects/standard/5183f151e4bb210219054160.png ASW0002pc9 0.0000874 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901ecc9.png ASW0001s10 0.9558911 34

http://spacewarps.org/subjects/standard/5183f151e4bb210219014424.png ASW0002194 0.0000001 43

http://spacewarps.org/subjects/standard/5183f151e4bb2102190172d8.png ASW0006nn7 0.0000001 26

http://spacewarps.org/subjects/standard/5183f151e4bb21021904bd33.png ASW0007do1 0.0006616 22

http://spacewarps.org/subjects/standard/5183f151e4bb2102190540f1.png ASW0007u4o 0.0000003 34

http://spacewarps.org/subjects/standard/5183f151e4bb210219059448.png ASW0006fv2 0.9985743 13

http://spacewarps.org/subjects/standard/5183f151e4bb2102190495ce.png ASW0004cn3 0.9937127 21

http://spacewarps.org/subjects/standard/5183f151e4bb2102190318ff.png ASW0004cn2 0.9972566 8

http://spacewarps.org/subjects/standard/5183f151e4bb2102190318fe.png ASW00055xa 0.9891456 11

http://spacewarps.org/subjects/standard/5183f151e4bb21021903ad3e.png ASW0008qe8 0.9989450 13

http://spacewarps.org/subjects/standard/5183f151e4bb2102190637a0.png ASW0006ouo 0.0000001 12

http://spacewarps.org/subjects/standard/5183f151e4bb21021904c350.png ASW0000f1h 0.9536387 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219004c25.png ASW0000f1i 0.0016157 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219004c26.png ASW0000bgu 0.0000002 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219003a0e.png ASW0000kut 0.0000056 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219006995.png ASW0004d3d 0.9872799 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219031b49.png ASW0005n6v 0.9972069 4

http://spacewarps.org/subjects/standard/5183f151e4bb2102190404a7.png ASW00025r0 0.9507088 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021901899c.png ASW0002i7l 0.9865697 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021901c8b1.png ASW0004pe2 0.2022948 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021903598a.png ASW00068qq 0.9520676 16

http://spacewarps.org/subjects/standard/5183f151e4bb2102190471c2.png ASW0000njn 0.9895404 20

http://spacewarps.org/subjects/standard/5183f151e4bb210219007733.png ASW0002qp8 0.9857259 18

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f3ac.png ASW0006bar 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219047eb3.png ASW000581b 0.9620308 3

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b7ef.png ASW0005ltt 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021903fdc1.png ASW0001u9h 0.9859180 12

http://spacewarps.org/subjects/standard/5183f151e4bb210219014f75.png ASW000879s 0.9932613 8

http://spacewarps.org/subjects/standard/5183f151e4bb21021905d6d0.png ASW00024pj 0.9986358 27

http://spacewarps.org/subjects/standard/5183f151e4bb210219018457.png ASW0001b78 0.0001102 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900eef4.png ASW0008wld 0.0006582 19

http://spacewarps.org/subjects/standard/5183f151e4bb210219065701.png ASW00014kt 0.0006146 43

http://spacewarps.org/subjects/standard/5183f151e4bb21021900cd6d.png ASW0005v3p 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219042cb5.png ASW0004qq8 0.9676146 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219036050.png ASW0008q5a 0.0003207 29

http://spacewarps.org/subjects/standard/5183f151e4bb21021906365e.png ASW0007soj 0.9921910 22

http://spacewarps.org/subjects/standard/5183f151e4bb210219058cf3.png ASW00048l5 0.9726988 22

http://spacewarps.org/subjects/standard/5183f151e4bb210219030479.png ASW000872e 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021905d5c6.png ASW00099fc 0.9897688 15

http://spacewarps.org/subjects/standard/5183f151e4bb2102190697f8.png ASW0007su4 0.9920828 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219058dbc.png ASW0001hgh 0.9977599 48

http://spacewarps.org/subjects/standard/5183f151e4bb210219010ea1.png ASW00077je 0.9630531 3

http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ea.png ASW00077jj 0.9668206 12

http://spacewarps.org/subjects/standard/5183f151e4bb2102190521ef.png ASW000662u 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219046446.png ASW0002vbp 0.0000001 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219020b15.png ASW0009bzf 0.1115790 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4eb.png ASW0000btm 0.9865619 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219003bda.png ASW0000qma 0.0177161 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190086c2.png ASW00084c0 0.0043379 30

http://spacewarps.org/subjects/standard/5183f151e4bb21021905c7f0.png ASW0000sk9 0.9752922 40

http://spacewarps.org/subjects/standard/5183f151e4bb210219009099.png ASW0005r7k 0.9516328 32

http://spacewarps.org/subjects/standard/5183f151e4bb210219041900.png ASW0005r7u 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021904190a.png ASW00068sy 0.0000001 11

http://spacewarps.org/subjects/standard/5183f151e4bb210219047212.png ASW0004evz 0.9881411 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021903245f.png ASW0001o7s 0.9897239 10

http://spacewarps.org/subjects/standard/5183f151e4bb2102190130d8.png ASW0004q57 0.9651999 24

http://spacewarps.org/subjects/standard/5183f151e4bb210219035d5b.png ASW0006k9m 0.9879590 16

http://spacewarps.org/subjects/standard/5183f151e4bb21021904ac1a.png ASW0001oci 0.9880044 31

http://spacewarps.org/subjects/standard/5183f151e4bb210219013182.png ASW0001akj 0.9831170 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021900ebc3.png ASW00099wr 0.9958514 11

http://spacewarps.org/subjects/standard/5183f151e4bb210219069a6b.png ASW0008snk 0.9674173 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219064310.png ASW0000ick 0.0000004 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219005ce4.png ASW0006jdr 0.9856068 19

http://spacewarps.org/subjects/standard/5183f151e4bb21021904a79f.png ASW000737p 0.9968255 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219050c05.png ASW000135f 0.9822759 35

http://spacewarps.org/subjects/standard/5183f151e4bb21021900c633.png ASW000402i 0.0763500 36

http://spacewarps.org/subjects/standard/5183f151e4bb21021902d95a.png ASW0004oo1 0.9969299 8

http://spacewarps.org/subjects/standard/5183f151e4bb2102190355e1.png ASW00054av 0.9719831 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021903a507.png ASW0006uhc 0.9917100 14

http://spacewarps.org/subjects/standard/5183f151e4bb21021904dfd0.png ASW00003iz 0.9937963 48

http://spacewarps.org/subjects/standard/5183f151e4bb2102190011db.png ASW000681u 0.0000001 15

http://spacewarps.org/subjects/standard/5183f151e4bb210219046e42.png ASW0001s7r 0.0014642 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219014517.png ASW000066w 0.9823131 9

http://spacewarps.org/subjects/standard/5183f151e4bb210219001f58.png ASW0007uos 0.0000273 23

http://spacewarps.org/subjects/standard/5183f151e4bb21021905971c.png ASW000000r 0.0000051 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900001b.png ASW0002lc9 0.9648556 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021901d889.png ASW00001yy 0.0000094 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190009fa.png ASW0004otg 0.0000002 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190356a4.png ASW0000hlt 0.0002223 1

http://spacewarps.org/subjects/standard/5183f151e4bb210219005921.png ASW00057pf 0.9933702 13

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b643.png ASW0002z3o 0.9584701 11

http://spacewarps.org/subjects/standard/5183f151e4bb210219021e34.png ASW0002a8r 0.9739114 14

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a05b.png ASW00036ez 0.9808452 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021902433b.png ASW00060wf 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219044a0f.png ASW0008xsq 0.0075074 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219065d1a.png ASW0008w0s 0.2460709 17

http://spacewarps.org/subjects/standard/5183f151e4bb21021906541c.png ASW0008fd1 0.0000129 24

http://spacewarps.org/subjects/standard/5183f151e4bb21021905ffc5.png ASW0009d4c 0.9793580 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021906aaac.png ASW0002ciz 0.9685346 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021901abeb.png ASW00078fo 0.9744614 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219052674.png ASW00078fs 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219052678.png ASW0003lw7 0.9520635 20

http://spacewarps.org/subjects/standard/5183f151e4bb210219029197.png ASW00024de 0.9975841 31

http://spacewarps.org/subjects/standard/5183f151e4bb2102190182a2.png ASW0000mnu 0.9778734 49

http://spacewarps.org/subjects/standard/5183f151e4bb2102190072ba.png ASW0000pjo 0.0000023 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219008154.png ASW0000mnt 0.0000090 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190072b9.png ASW0007thm 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021905910a.png ASW0002ro1 0.9889915 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f891.png ASW0008384 0.3350917 16

http://spacewarps.org/subjects/standard/5183f151e4bb21021905c254.png ASW00008da 0.9864686 13

http://spacewarps.org/subjects/standard/5183f151e4bb210219002a5e.png ASW00071xq 0.9758661 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021905058e.png ASW0006vyq 0.0000034 42

http://spacewarps.org/subjects/standard/5183f151e4bb21021904e752.png ASW0002emw 0.9850455 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b698.png ASW000083g 0.0000001 51

http://spacewarps.org/subjects/standard/5183f151e4bb2102190028fc.png ASW0006frk 0.9893204 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219049550.png ASW0006c3z 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb2102190482cf.png ASW00033sc 0.9921568 7

http://spacewarps.org/subjects/standard/5183f151e4bb2102190235ec.png ASW0002b9r 0.9950075 47

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a58f.png ASW00033g4 0.0000004 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219023434.png ASW00039up 0.9946309 13

http://spacewarps.org/subjects/standard/5183f151e4bb2102190254a1.png ASW00029c6 0.0008550 46

http://spacewarps.org/subjects/standard/5183f151e4bb210219019bc6.png ASW0008360 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021905c208.png ASW0005eom 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021903d996.png ASW00002vo 0.0000003 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219000e94.png ASW0005r4n 0.9974458 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219041897.png ASW0000wy6 0.0117687 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900a6ce.png ASW0005abs 0.9599478 10

http://spacewarps.org/subjects/standard/5183f151e4bb21021903c388.png ASW0001hoz 0.9812980 12

http://spacewarps.org/subjects/standard/5183f151e4bb210219010fd3.png ASW0009cd5 0.0838328 26

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d9.png ASW0009ccx 0.0043871 23

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a6d1.png ASW0000wbs 0.0057063 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900a3a8.png ASW000990p 0.0000001 10

http://spacewarps.org/subjects/standard/5183f151e4bb2102190695e9.png ASW0003gj8 0.9963639 15

http://spacewarps.org/subjects/standard/5183f151e4bb210219027674.png ASW0005s12 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219041d26.png ASW0001az2 0.0000001 29

http://spacewarps.org/subjects/standard/5183f151e4bb21021900edce.png ASW0002wet 0.9804352 24

http://spacewarps.org/subjects/standard/5183f151e4bb210219021095.png ASW00001e1 0.9973884 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219000709.png ASW0004vhx 0.9951441 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219037875.png ASW0004vht 0.9751420 10

http://spacewarps.org/subjects/standard/5183f151e4bb210219037871.png ASW00067ov 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219046c6f.png ASW0001sl1 0.9928866 13

http://spacewarps.org/subjects/standard/5183f151e4bb2102190146f5.png ASW00086wv 0.0000581 19

http://spacewarps.org/subjects/standard/5183f151e4bb21021905d4ff.png ASW00098nt 0.9724689 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219069419.png ASW00099v2 0.0004023 26

http://spacewarps.org/subjects/standard/5183f151e4bb210219069a2e.png ASW00016cn 0.9663391 44

http://spacewarps.org/subjects/standard/5183f151e4bb21021900d667.png ASW0005no8 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb210219040718.png ASW00078ni 0.0034764 23

http://spacewarps.org/subjects/standard/5183f151e4bb21021905278e.png ASW000091m 0.9663804 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219002dca.png ASW0001mje 0.0614669 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901285a.png ASW0007061 0.0000017 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021904fc99.png ASW0005wl9 0.9958300 12

http://spacewarps.org/subjects/standard/5183f151e4bb21021904343d.png ASW0000gs3 0.0000079 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190054f3.png ASW00062xw 0.9621585 19

http://spacewarps.org/subjects/standard/5183f151e4bb210219045464.png ASW0001fpx 0.9891655 8

http://spacewarps.org/subjects/standard/5183f151e4bb2102190105d5.png ASW0008h6s 0.9941180 9

http://spacewarps.org/subjects/standard/5183f151e4bb210219060904.png ASW0001dk1 0.9905595 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021900fae1.png ASW0007ooc 0.0001410 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190578ac.png ASW0006v69 0.9769766 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021904e351.png ASW0009cnk 0.9833248 11

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a850.png ASW0000boq 0.9591860 29

http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2a.png ASW0000bor 0.0032814 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219003b2b.png ASW0001scq 0.9819256 14

http://spacewarps.org/subjects/standard/5183f151e4bb2102190145ca.png ASW0005ph2 0.9575726 3

http://spacewarps.org/subjects/standard/5183f151e4bb210219041036.png ASW0001brs 0.0000613 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900f1d8.png ASW0002qrt 0.2227508 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f409.png ASW0008ct1 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021905f2d5.png ASW0000wf4 0.0000001 25

http://spacewarps.org/subjects/standard/5183f151e4bb21021900a420.png ASW00038cr 0.9939420 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0b.png ASW00038cq 0.9865079 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219024d0a.png ASW0009ayk 0.0000450 19

http://spacewarps.org/subjects/standard/5183f151e4bb210219069fbc.png ASW0000isd 0.9945940 32

http://spacewarps.org/subjects/standard/5183f151e4bb210219005f1d.png ASW0003cpv 0.9703087 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219026323.png ASW00002ng 0.0000010 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219000d6c.png ASW0008dxt 0.0048544 25

http://spacewarps.org/subjects/standard/5183f151e4bb21021905f891.png ASW0003q90 0.9934436 4

http://spacewarps.org/subjects/standard/5183f151e4bb21021902a7a4.png ASW00069fm 0.9983546 30

http://spacewarps.org/subjects/standard/5183f151e4bb210219047542.png ASW0008xjb 0.0139223 27

http://spacewarps.org/subjects/standard/5183f151e4bb210219065bc7.png ASW0000vp4 0.0007258 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900a078.png ASW0004lms 0.9949704 19

http://spacewarps.org/subjects/standard/5183f151e4bb210219034684.png ASW0005rmr 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219041b23.png ASW00016zj 0.9574544 24

http://spacewarps.org/subjects/standard/5183f151e4bb21021900d99f.png ASW0000i98 0.9501395 30

http://spacewarps.org/subjects/standard/5183f151e4bb210219005c6c.png ASW0000or0 0.9793596 26

http://spacewarps.org/subjects/standard/5183f151e4bb210219007d4c.png ASW0006tfw 0.9582275 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021904da8c.png ASW0002tow 0.9768473 14

http://spacewarps.org/subjects/standard/5183f151e4bb2102190202d0.png ASW000080k 0.9872287 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219002894.png ASW0000z9l 0.9579909 16

http://spacewarps.org/subjects/standard/5183f151e4bb21021900b289.png ASW000018a 0.0000038 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900063a.png ASW000375u 0.9982698 27

http://spacewarps.org/subjects/standard/5183f151e4bb210219024702.png ASW0007l8i 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219056742.png ASW00057c4 0.0000001 10

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b464.png ASW00026d6 0.9917335 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219018cba.png ASW0002eb2 0.0014966 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4ee.png ASW0002eau 0.0020410 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901b4e6.png ASW00099o7 0.9586766 26

http://spacewarps.org/subjects/standard/5183f151e4bb210219069937.png ASW000392k 0.9965401 7

http://spacewarps.org/subjects/standard/5183f151e4bb2102190250ac.png ASW0005nhm 0.9638209 30

http://spacewarps.org/subjects/standard/5183f151e4bb21021904062a.png ASW0000gz9 0.9931312 16

http://spacewarps.org/subjects/standard/5183f151e4bb2102190055f5.png ASW0000gzh 0.0000131 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190055fd.png ASW00053f7 0.9924378 12

http://spacewarps.org/subjects/standard/5183f151e4bb21021903a093.png ASW0001dt1 0.9567922 23

http://spacewarps.org/subjects/standard/5183f151e4bb21021900fc25.png ASW00053fj 0.9748313 49

http://spacewarps.org/subjects/standard/5183f151e4bb21021903a09f.png ASW0002j4d 0.9546089 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021901cd4d.png ASW0000nn4 0.9796065 39

http://spacewarps.org/subjects/standard/5183f151e4bb2102190077b0.png ASW0006c9a 0.9886119 11

http://spacewarps.org/subjects/standard/5183f151e4bb21021904838e.png ASW0002oz7 0.0000176 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901eaf3.png ASW0003agn 0.9668959 43

http://spacewarps.org/subjects/standard/5183f151e4bb2102190257b7.png ASW0001m4l 0.9879547 49

http://spacewarps.org/subjects/standard/5183f151e4bb210219012645.png ASW0000fyd 0.9648816 71

http://spacewarps.org/subjects/standard/5183f151e4bb2102190050c5.png ASW000524x 0.9656719 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219039a11.png ASW0005252 0.9844611 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219039a16.png ASW00017ls 0.0000905 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900dcc0.png ASW0006wdu 0.9981160 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021904e972.png ASW0000bv8 0.0000233 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219003c14.png ASW0002xve 0.9945064 7

http://spacewarps.org/subjects/standard/5183f151e4bb2102190217fa.png ASW0002xv6 0.9832091 5

http://spacewarps.org/subjects/standard/5183f151e4bb2102190217f2.png ASW0002p5v 0.9906009 13

http://spacewarps.org/subjects/standard/5183f151e4bb21021901ebe3.png ASW0009a19 0.9528699 4

http://spacewarps.org/subjects/standard/5183f151e4bb210219069b0d.png ASW0002r06 0.9940529 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f536.png ASW0002r08 0.9821927 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f538.png ASW00079wv 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219052def.png ASW000260c 0.9847498 8

http://spacewarps.org/subjects/standard/5183f151e4bb210219018aec.png ASW000260e 0.0001297 37

http://spacewarps.org/subjects/standard/5183f151e4bb210219018aee.png ASW0007xqc 0.0000001 14

http://spacewarps.org/subjects/standard/5183f151e4bb21021905a684.png ASW0000u37 0.9922268 41

http://spacewarps.org/subjects/standard/5183f151e4bb210219009853.png ASW0000u3a 0.9973031 33

http://spacewarps.org/subjects/standard/5183f151e4bb210219009856.png ASW0005jbt 0.9778492 27

http://spacewarps.org/subjects/standard/5183f151e4bb21021903f119.png ASW0007xqm 0.9924636 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021905a68e.png ASW0000jj0 0.0688318 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190062dc.png ASW00012lt 0.9931875 17

http://spacewarps.org/subjects/standard/5183f151e4bb21021900c371.png ASW0009bcx 0.0000002 17

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a1c1.png ASW00048u4 0.9742897 42

http://spacewarps.org/subjects/standard/5183f151e4bb2102190305bc.png ASW0006e3j 0.9971356 9

http://spacewarps.org/subjects/standard/5183f151e4bb210219048cdf.png ASW0002ysy 0.9772258 19

http://spacewarps.org/subjects/standard/5183f151e4bb210219021cb2.png ASW0003hvz 0.9855299 15

http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4f.png ASW0003hvx 0.9822340 14

http://spacewarps.org/subjects/standard/5183f151e4bb210219027d4d.png ASW0002yta 0.9957761 32

http://spacewarps.org/subjects/standard/5183f151e4bb210219021cbe.png ASW0007416 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021905102a.png ASW0007pwb 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219057edb.png ASW00056ut 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b1f5.png ASW000282w 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219019568.png ASW0000qpl 0.9529357 37

http://spacewarps.org/subjects/standard/5183f151e4bb210219008739.png ASW0000wv0 0.0000020 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900a65c.png ASW0009bzo 0.9612347 10

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4f4.png ASW0007yn2 0.0755943 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021905ab1e.png ASW0009bzz 0.9985911 4

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4ff.png ASW0009bzw 0.9844175 7

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a4fc.png ASW0000ljc 0.9616797 28

http://spacewarps.org/subjects/standard/5183f151e4bb210219006d08.png ASW0009d1u 0.0765714 24

http://spacewarps.org/subjects/standard/5183f151e4bb21021906aa52.png ASW00050dz 0.9951869 13

http://spacewarps.org/subjects/standard/5183f151e4bb210219039137.png ASW00057gw 0.9965523 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021903b510.png ASW0002b9s 0.9962135 6

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a590.png ASW0001x77 0.9951692 13

http://spacewarps.org/subjects/standard/5183f151e4bb210219015e53.png ASW0000q0z 0.9682742 35

http://spacewarps.org/subjects/standard/5183f151e4bb2102190083c3.png ASW0005wi0 0.0000001 8

http://spacewarps.org/subjects/standard/5183f151e4bb2102190433c8.png ASW000251j 0.9983329 31

http://spacewarps.org/subjects/standard/5183f151e4bb210219018607.png ASW0001771 0.9767806 37

http://spacewarps.org/subjects/standard/5183f151e4bb21021900daad.png ASW000390p 0.0000119 25

http://spacewarps.org/subjects/standard/5183f151e4bb210219025069.png ASW0001vs0 0.9704887 29

http://spacewarps.org/subjects/standard/5183f151e4bb210219015720.png ASW0001m2l 0.9916034 7

http://spacewarps.org/subjects/standard/5183f151e4bb2102190125fd.png ASW00017cn 0.9953570 18

http://spacewarps.org/subjects/standard/5183f151e4bb21021900db77.png ASW00078qk 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb2102190527fc.png ASW0006bb7 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219047ec3.png ASW00008yb 0.9621877 33

http://spacewarps.org/subjects/standard/5183f151e4bb210219002d53.png ASW0002bry 0.9944684 12

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a81e.png ASW00054sh 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021903a781.png ASW000098v 0.9557436 16

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ecf.png ASW000098n 0.0000062 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec7.png ASW000098o 0.3888287 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219002ec8.png ASW0007dih 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219054029.png ASW0005wxi 0.0000001 11

http://spacewarps.org/subjects/standard/5183f151e4bb2102190435f6.png ASW0002rx1 0.9991815 9

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f9d5.png ASW0003npm 0.0007995 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219029aca.png ASW0005zg1 0.0040780 42

http://spacewarps.org/subjects/standard/5183f151e4bb2102190442b1.png ASW00023vf 0.9771071 24

http://spacewarps.org/subjects/standard/5183f151e4bb21021901801b.png ASW0009cb8 0.9842217 23

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a694.png ASW0000336 0.1178867 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219000fa2.png ASW0002g4u 0.0026618 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901be2e.png ASW0001c6q 0.9702514 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021900f3f2.png ASW0006eqq 0.0000234 45

http://spacewarps.org/subjects/standard/5183f151e4bb210219049022.png ASW00018s1 0.0000015 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021900e2b1.png ASW0002r6u 0.9809996 13

http://spacewarps.org/subjects/standard/5183f151e4bb21021901f626.png ASW0004t3g 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219036c4c.png ASW0006eix 0.9889984 23

http://spacewarps.org/subjects/standard/5183f151e4bb210219048f09.png ASW00037wf 0.0000001 15

http://spacewarps.org/subjects/standard/5183f151e4bb210219024abf.png ASW0007f70 0.9870621 42

http://spacewarps.org/subjects/standard/5183f151e4bb2102190548ac.png ASW0000ccd 0.0000557 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219003e7d.png ASW0007xs6 0.9778325 14

http://spacewarps.org/subjects/standard/5183f151e4bb21021905a6c6.png ASW0002ifr 0.0001891 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901c9d7.png ASW00007mq 0.9919367 43

http://spacewarps.org/subjects/standard/5183f151e4bb2102190026a2.png ASW0009baa 0.0002902 15

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a162.png ASW0009bal 0.0000001 4

http://spacewarps.org/subjects/standard/5183f151e4bb21021906a16d.png ASW00063k0 0.9591609 17

http://spacewarps.org/subjects/standard/5183f151e4bb210219045780.png ASW0000gau 0.9604534 22

http://spacewarps.org/subjects/standard/5183f151e4bb210219005286.png ASW0004e7g 0.9845510 24

http://spacewarps.org/subjects/standard/5183f151e4bb2102190320ec.png ASW0009anl 0.9886676 22

http://spacewarps.org/subjects/standard/5183f151e4bb210219069e31.png ASW0008tp2 0.0000001 7

http://spacewarps.org/subjects/standard/5183f151e4bb210219064856.png ASW0002o7b 0.9825282 10

http://spacewarps.org/subjects/standard/5183f151e4bb21021901e707.png ASW0004p85 0.0278396 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190358b5.png ASW0002cu0 0.9898176 4

http://spacewarps.org/subjects/standard/5183f151e4bb21021901ad78.png ASW00009n7 0.9731329 26

http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d3.png ASW00009n9 0.1452734 50

http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d5.png ASW00009n8 0.9538296 35

http://spacewarps.org/subjects/standard/5183f151e4bb2102190030d4.png ASW0000a6f 0.7541436 50

http://spacewarps.org/subjects/standard/5183f151e4bb210219003387.png ASW00098cz 0.0000017 27

http://spacewarps.org/subjects/standard/5183f151e4bb210219069293.png ASW00098db 0.0145911 33

http://spacewarps.org/subjects/standard/5183f151e4bb21021906929f.png ASW0004o2p 0.0000004 38

http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e1.png ASW0004o2t 0.9563873 3

http://spacewarps.org/subjects/standard/5183f151e4bb2102190352e5.png ASW0008ujv 0.0000001 6

http://spacewarps.org/subjects/standard/5183f151e4bb210219064cab.png ASW00061bt 0.0000009 42

http://spacewarps.org/subjects/standard/5183f151e4bb210219044c39.png ASW0006rt2 0.0000001 5

http://spacewarps.org/subjects/standard/5183f151e4bb21021904d246.png ASW0003lku 0.9721485 4

http://spacewarps.org/subjects/standard/5183f151e4bb210219028ffe.png ASW0000aab 0.0002544 51

http://spacewarps.org/subjects/standard/5183f151e4bb210219003413.png ASW000269w 0.9983154 29

http://spacewarps.org/subjects/standard/5183f151e4bb210219018c44.png ASW000999g 0.0149549 27

http://spacewarps.org/subjects/standard/5183f151e4bb210219069724.png ASW0002apb 0.0002044 50

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a2af.png ASW0008vsd 0.9988715 9

http://spacewarps.org/subjects/standard/5183f151e4bb2102190652ed.png ASW0003997 0.9746200 8

http://spacewarps.org/subjects/standard/5183f151e4bb21021902519b.png ASW0008bfg 0.9576523 20

http://spacewarps.org/subjects/standard/5183f151e4bb21021905ebdc.png ASW0002bwy 0.9927501 22

http://spacewarps.org/subjects/standard/5183f151e4bb21021901a8d2.png ASW0002xmt 0.9794185 17

http://spacewarps.org/subjects/standard/5183f151e4bb2102190216c5.png ASW0007dqp 0.0000001 8

http://spacewarps.org/subjects/standard/5183

[anupreeta27]

ASW0001e2j is CFHTLS_004_0843 meaning not D11. this has both sims and a real lens candidate. http://talk.spacewarps.org/#/subjects/ASW000993u this is the same image without sims and this should be D11 ie. - CFHTLS_204_0843

[aprajita]

I still don't understand how ASW0001e2j is CFHTLS_004_0843 with two lots of very obvious sims, and a real lens, ends up in the rejected pile...

[drphilmarshall]

I smell a rat! Or at least a bug. Those subjects shouldn't be being rejected by SWAP at all, because they should be labelled as training images, so never retired...

I made a mistake - I counted all lenses from Anu's finknwnlenscat file that did not appear in the candidate list from SWAP as failures, whereas actually all of those that had sims placed in them are indeed excluded form the SWAP candidate list - correctly, because they are not test subjects! So the problem is that the lens cat has the wrong zooids in it: I need a catalog that contains ASW000993u instead of ASW0001e2j, and so on - the D11 zooids. Can you help, please Anu? Then I'll recompute the completenesses (which will be better!) and remake the galleries.

BTW, ASW000993u was detected by the community, with P=0.9799091 :-)

On Wed, Dec 4, 2013 at 10:27 AM, aprajita notifications@github.com wrote:

I still don't understand how ASW0001e2j is CFHTLS_004_0843 with two lots of very obvious sims, and a real lens, ends up in the rejected pile...

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29813916 .

[anupreeta27]

hi phil,

this catalog gives all ids for all datasets, as the name suggests http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11

everything that has ids of this format CFHTLS[2-3]??* belongs to D11 cheers, anu

On Thu, Dec 5, 2013 at 12:35 PM, Phil Marshall notifications@github.comwrote:

I smell a rat! Or at least a bug. Those subjects shouldn't be being rejected by SWAP at all, because they should be labelled as training images, so never retired...

I made a mistake - I counted all lenses from Anu's finknwnlenscat file that did not appear in the candidate list from SWAP as failures, whereas actually all of those that had sims placed in them are indeed excluded form the SWAP candidate list - correctly, because they are not test subjects! So the problem is that the lens cat has the wrong zooids in it: I need a catalog that contains ASW000993u instead of ASW0001e2j, and so on - the D11 zooids. Can you help, please Anu? Then I'll recompute the completenesses (which will be better!) and remake the galleries.

BTW, ASW000993u was detected by the community, with P=0.9799091 :-)

On Wed, Dec 4, 2013 at 10:27 AM, aprajita notifications@github.com wrote:

I still don't understand how ASW0001e2j is CFHTLS_004_0843 with two lots of very obvious sims, and a real lens, ends up in the rejected pile...

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29813916>

.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29869068 .

[drphilmarshall]

OK! I checked to make sure I took the CFHTLS_2* or 3* version in stead of 0* or 1*, and get the following new numbers:

P > 0.95: completeness = 68% P > 2e-7: completeness = 74%

Remaking the galleries now...

On Wed, Dec 4, 2013 at 10:39 PM, anupreeta27 notifications@github.comwrote:

hi phil,

this catalog gives all ids for all datasets, as the name suggests http://member.ipmu.jp/anupreeta.more/listfiles_d1_d11

everything that has ids of this format CFHTLS[2-3]??* belongs to D11 cheers, anu

On Thu, Dec 5, 2013 at 12:35 PM, Phil Marshall notifications@github.comwrote:

I smell a rat! Or at least a bug. Those subjects shouldn't be being rejected by SWAP at all, because they should be labelled as training images, so never retired...

I made a mistake - I counted all lenses from Anu's finknwnlenscat file that did not appear in the candidate list from SWAP as failures, whereas actually all of those that had sims placed in them are indeed excluded form the SWAP candidate list - correctly, because they are not test subjects! So the problem is that the lens cat has the wrong zooids in it: I need a catalog that contains ASW000993u instead of ASW0001e2j, and so on - the D11 zooids. Can you help, please Anu? Then I'll recompute the completenesses (which will be better!) and remake the galleries.

BTW, ASW000993u was detected by the community, with P=0.9799091 :-)

On Wed, Dec 4, 2013 at 10:27 AM, aprajita notifications@github.com wrote:

I still don't understand how ASW0001e2j is CFHTLS_004_0843 with two lots of very obvious sims, and a real lens, ends up in the rejected pile...

— Reply to this email directly or view it on GitHub<

https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29813916>

.

— Reply to this email directly or view it on GitHub< https://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29869068>

.

— Reply to this email directly or view it on GitHubhttps://github.com/drphilmarshall/SpaceWarps/issues/9#issuecomment-29869190 .

[drphilmarshall]

Closing this for now: it'll get re-opened as we make plots for the papers.