search

ratt-ru / QuartiCal

CubiCal, but with greater power.
MIT License
8 stars 4 forks source link

Need some sort of summary chi-sq output #96

Closed o-smirnov closed 2 years ago

o-smirnov commented 3 years ago

This bit of cubical's output log...

# INFO      21:22:16 - main               [8.4/75.8 23.5/600.8 0.0Gb]   summary stats for {chi2:.3f}:
#              D0T0F0  D0T1F0  D0T2F0  D0T3F0  D0T4F0  D0T5F0  D0T6F0  D0T7F0
#   D0T0F0     1.543   1.538   1.535   1.540   1.541   1.538   1.539   1.539

...was extremely useful (to quickly tell if a calibration was going to plan or not). It would be nice to have it back in QuartiCal.

JSKenyon commented 3 years ago

I know that we need something like this, I just haven't yet clicked on how best to do it when using dask. I actually do have some (currently commented) code that was producing pre and post solve chi-squared values. I will get back to it at some point - I hated my original implementation but I am more familiar with dask now.

A really cool solution would be to implement this as a form of "progress bar" that dynamically updates as chunks finishing processing, but I am not 100% sure how to wrangle that.

sjperkins commented 3 years ago

A really cool solution would be to implement this as a form of "progress bar" that dynamically updates as chunks finishing processing, but I am not 100% sure how to wrangle that.

I made one of these for crystalball -- it's in codex:

https://github.com/ska-sa/codex-africanus/blob/master/africanus/util/dask_util.py#L149

JSKenyon commented 3 years ago

Thanks @sjperkins - I haven't looked at this in detail, but is it possible to pull values out of tasks?

sjperkins commented 3 years ago

Thanks @sjperkins - I haven't looked at this in detail, but is it possible to pull values out of tasks?

No it doesn't take results into account, merely tasks completed.

JSKenyon commented 3 years ago

Yeah - while I definitely like the idea of a better progress bar, what we need here is a little closer to logging. I will continue thinking about it for now - I have the beginnings of an idea.

o-smirnov commented 3 years ago

A progress bar (or at least some form of progress) was the next thing on my wishlist. Do you want a separate discussion for that?

The underlying solution is the same I guess. You want to take summary statistics when a chunk is done, and you want to propagate them back out to the main thread somehow. Based on that you can have both progress reporting, and final stats...

JSKenyon commented 2 years ago

Closing this for now - there is now logging as each chunk is completed as well as a summary at the end. I have abandoned progress bars for now, as they are can be very misleading. I will keep them in mind, but it seems like wasted effort for now.