search

cta-observatory / iact_event_types

BSD 3-Clause "New" or "Revised" License
4 stars 7 forks source link

Look again at train statistics #51

Closed orelgueta closed 2 years ago

orelgueta commented 2 years ago

While I was looking at train statistics of the inner camera I also took a look with the full camera, plot is below.

scores_comparison_1

Looks like we can always gain from more statistics, but the gain isn't super significant and we can stay with 25% for now. Hopefully in the future we will simply simulate more diffuse gammas. Thoughts?

TarekHC commented 2 years ago

Hi @orelgueta

I essentially agree with you: it is good news that we still improve with better statistics (as in the future, we will probably have better statistics).

When going from 15 to 25% we gain almost the same factor than when going from 25 to 75%. So leaving a 25% statistics for this step feels reasonable to me (and consistent with what Juan found independently, which is cool).

@JBernete did you produce these plots as a function of the offset angle? I don't remember anymore. I would personally like to have these plots produced for 0-1, 1-2, 2-3 & 3-4 deg in offset. Performance may be worse at larger offsets, but as long as we see the same evolution with respect to statistics, I would choose 25% for the time being and close the issue (until we go to a new production, and we need to re-evaluate it again!).

JBernete commented 2 years ago

Yes @TarekHC, I produced that for the different offset rings. I'm updating my fork, as I am still not able to update a branch in this repository. So I'll make a new PR with the code I used to produce these plots. At the moment I only have these plots for 25% train. In a few minutes I'll be able to upload them with different train statistics. I'll upload the ones that I have so far to Issue #40 as I think it's more appropiate than here.

TarekHC commented 2 years ago

Ok, as we close #40, I feel it also makes sense to close this issue. 25% it is!