search

ecmwf / ecpoint-calibrate

Interactive GUI (developed in Python) for calibration and conditional verification of numerical weather prediction model outputs.
GNU General Public License v3.0
21 stars 8 forks source link

Statistics available also for the predictors #133

Closed EstiGascon closed 3 years ago

EstiGascon commented 3 years ago

A really useful tool would be to be able to check the predictors and predictant values as well as we can check the minimum, maximum and mean values when we load the FE or FER table. It would allow as to have a fast look to the database, check for possible errors and have a general idea of the values of the predictors. That would probably be useful in both modules, in Module 1 and Module 2.

FatimaPillosu commented 3 years ago

This was discussed long time ago with Anirudha, but it didn't seem necessary as we could have a look to the ascii point data table. However, now with the parquet file this would not be possible anymore. @onyb, would it be possible to reconsider this idea again?

onyb commented 3 years ago

If you know which column to look for precisely (which seems to be the case, i.e., FE/FER), we can display a preview for both ASCII and Parquet files. One thing to note is that the Parquet file can have millions of rows, so showing all the rows is not practical.

By the way, aren't we already displaying the min and max values? See screenshot below:

Screenshot 2020-12-11 at 02 20 09
EstiGascon commented 3 years ago

Many thanks for the answer Anirudha. What we referred is to see a display of the min and max for the predictors and the predictant, not the FE/FER. I put as example FER because it was already done for that variable, so you could see what we wanted.

onyb commented 3 years ago

Fixed in v0.17.0. Check out the screenshots in the release notes for more info, and feel free to reopen this issue if it needs improvements.