Warnings have been turned off

[aidanheerdegen]

Capturing of warnings is turned on here:

https://github.com/COSIMA/cosima-cookbook/blob/19808e091c18f7c7b98a24abafe9e7f6807bbd6a/cosima_cookbook/database.py#L45

It occurred in this commit https://github.com/COSIMA/cosima-cookbook/commit/c1968611ec0b9c0b7ac09ff088745ce27817c3c8

It means users no longer get warnings that were designed to tell users there was potentially a problem with their query:

https://github.com/COSIMA/cosima-cookbook/blob/19808e091c18f7c7b98a24abafe9e7f6807bbd6a/cosima_cookbook/querying.py#L307-L322

Arguably the multiple filenames warning is no longer fit for purpose. Since the diagnostics have been split out into individual files with dates in the filename this warning is misleading and unhelpful.

However there is currently an issue with queries returning incorrect data (#252) and warning the user about this might be one potential approach.

[angus-g]

Agreed about the multiple filename warning, but maybe we should be showing "warn" level warnings by default anyway? It's curious that they'd be hidden!

[aidanheerdegen]

Agree definitely should be emitting warnings.

Weirdly in #252 I am getting warnings. Hrm. Might have jumped the gun on this. Will check more thoroughly tomorrow.

[aidanheerdegen]

They're being captured by logging due to the logging.captureWarnings(True) in database.py.

Simple example:

import cosima_cookbook as cc
from cosima_cookbook import querying

import warnings
import logging

logging.captureWarnings(False)
warnings.simplefilter("always")

session = cc.database.create_session()

# Define experiment in database
expt = '01deg_jra55v140_iaf_cycle2'
start_time = '1990-01-01'
end_time   = '1990-12-31'

files = querying._ncfiles_for_variable(expt,
                                       'u',
                                       session,
                                       start_time=start_time,
                                       end_time=end_time,
                                       frequency='1 monthly',
                                       attrs_unique={'cell_methods': 'area: time: mean'},
                                       )

With the latest version of the cookbook I get this output (it loads the files because I've reset the warnings filter, otherwise it would thrown an exception which does make it to the console)

$ python test.py 
/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.07/lib/python3.9/site-packages/cosima_cookbook/querying.py:357: QueryWarning: Your query returns variables from files with different cell_methods: {'time: mean', 'time: mean_pow(02)'}. This could lead to unexpected behaviour! Disambiguate by passing attrs={'cell_methods'=''} to getvar, specifying the desired attribute value.
  warnings.warn(
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-mean-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-pow02-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-mean-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-mean-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-mean-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-mean-ym_1990_10.nc

If I comment out logging.captureWarnings(False) no warning makes it to stderr

$ python test.py 
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-mean-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output371/ocean/ocean-3d-u-1-monthly-pow02-ym_1989_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output372/ocean/ocean-3d-u-1-monthly-mean-ym_1990_01.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-mean-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output373/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_04.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output374/ocean/ocean-3d-u-1-monthly-mean-ym_1990_07.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-pow02-ym_1990_10.nc
/g/data/cj50/access-om2/raw-output/access-om2-01/01deg_jra55v140_iaf_cycle2/output375/ocean/ocean-3d-u-1-monthly-mean-ym_1990_10.nc

Consequently I think this should be removed from database.py.

[aidanheerdegen]

Created a custom warning for query errors and reset filter for it to throw exception in https://github.com/COSIMA/cosima-cookbook/pull/257