Open tlvu opened 6 months ago
Only merge if https://github.com/bird-house/birdhouse-deploy/pull/413 can be merged.
This is mostly to have a nice notebook diff.
_ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb::Cell 0 _ Notebook cell execution failed Cell 0: Cell outputs differ Input: from siphon.catalog import TDSCatalog url = "https://boreas.ouranos.ca/twitcher/ows/proxy/thredds/catalog/datasets/simulations/bias_adjusted/cmip5/ouranos/cb-oura-1.0/catalog.xml" # TEST_USE_PROD_DATA # Create Catalog cat = TDSCatalog(url) # List of datasets print(f"Number of datasets: {len(cat.datasets)}") # Access mechanisms - here we are interested in OPENDAP, a data streaming protocol cds = cat.datasets[0] print(f"Access URLs: {tuple(cds.access_urls.keys())}") Traceback: mismatch 'stdout' assert reference_output == test_output failed: "Number of da...cdfSubset')\n" == "Number of da...cdfSubset')\n" Skipping 43 identical leading characters in diff, use -v to show Skipping 50 identical trailing characters in diff, use -v to show - erver', 'OpenDAP', 'NC ? ^^^ + erver', 'OPENDAP', 'NC ? ^^^ _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb::Cell 2 _ Notebook cell execution failed Cell 2: Cell outputs differ Input: # Extract a subset of the file # Again, this only creates an in-memory representation of the data sub = ds.pr.sel(time="2050").sel(lon=-80, lat=46, method="nearest") # The data is only downloaded when we actuall need it for a computation. sub.mean(keep_attrs=True).compute() Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...[256 16 16]' == '<xarray.Data...[256 16 16]' Skipping 238 identical trailing characters in diff, use -v to show <xarray.DataArray 'pr' ()> - array(4.372211e-05, dty + array(3.975487e-05, dty _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb::Cell 3 _ Notebook cell execution failed Cell 3: Cell outputs differ Input: rcp45_data = [cat.datasets[x] for x in cat.datasets if "rcp45" in x] rcp45_data Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '[day_NorESM1...50-2100.ncml]' == '[day_ACCESS1...50-2100.ncml]' Skipping 66 identical trailing characters in diff, use -v to show - [day_ACCESS1-3_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, ? ^^^ ^ ^ + [day_NorESM1-M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, ? ^^^ ^ ^ + day_MPI-ESM-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_IPSL-CM5B-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_IPSL-CM5A-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_INM-CM4_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_HadGEM2-CC_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_GFDL-ESM2M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_CanESM2_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_CMCC-CMS_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, day_BNU-ESM_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_CMCC-CMS_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_CanESM2_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_GFDL-ESM2M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_HadGEM2-CC_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_INM-CM4_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_IPSL-CM5A-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_IPSL-CM5B-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_MPI-ESM-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_NorESM1-M_historic ? ^^^ ^ ^ + day_ACCESS1-3_historic ? ^^^ ^ ^ _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb::Cell 5 _ Notebook cell execution failed Cell 5: Cell outputs differ Input: datasets = {} for r in ["rcp45", "rcp85"]: expected = 11 datasets[r] = get_ncfilelist(r, expected, url) # Display search results display(datasets["rcp45"]) display(datasets["rcp85"]) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '[day_NorESM1...50-2100.ncml]' == '[day_ACCESS1...50-2100.ncml]' Skipping 66 identical trailing characters in diff, use -v to show - [day_ACCESS1-3_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, ? ^^^ ^ ^ + [day_NorESM1-M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, ? ^^^ ^ ^ + day_MPI-ESM-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_IPSL-CM5B-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_IPSL-CM5A-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_INM-CM4_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_HadGEM2-CC_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_GFDL-ESM2M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_CanESM2_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, + day_CMCC-CMS_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, day_BNU-ESM_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_CMCC-CMS_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_CanESM2_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_GFDL-ESM2M_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_HadGEM2-CC_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_INM-CM4_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_IPSL-CM5A-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_IPSL-CM5B-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_MPI-ESM-LR_historical+rcp45_r1i1p1_na10kgrid_qm-moving-50bins-detrend_1950-2100.ncml, - day_NorESM1-M_historic ? ^^^ ^ ^ + day_ACCESS1-3_historic ? ^^^ ^ ^ _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 0 _ Notebook cell execution failed Cell 0: Cell outputs differ Input: import warnings import xarray as xr from IPython.display import display # Fancy representation of xarray objects from siphon.catalog import TDSCatalog warnings.simplefilter("ignore") url = "https://boreas.ouranos.ca/twitcher/ows/proxy/thredds/catalog/datasets/simulations/bias_adjusted/cmip5/ouranos/cb-oura-1.0/catalog.xml" # TEST_USE_PROD_DATA # Create Catalog cat = TDSCatalog(url) # DAP link for this demo ds_url = cat.datasets[0].access_urls["OPENDAP"] # xarray.Dataset ds = xr.open_dataset(ds_url, chunks=dict(time=256 * 2, lon=32, lat=32)) display(ds) a = ds.tasmin.isel(time=0).plot(figsize=(10, 4)) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data... ACCESS1-3' Skipping 50 identical leading characters in diff, use -v to show - time: 55152) ? - + time: 55115) ? + Coordinates: * lat (lat) float32 66.62 66.54 66.46 66.37 ... 40.29 40.21 40.12 40.04 * lon (lon) float32 -120.8 -120.7 -120.6 -120.5 ... -54.63 -54.55 -54.46 - * time (time) datetime64[ns]/DATE/1950-01-02 ... 2100-12-31 + * time (time) object/DATE/TIME/.../DATE/00:00:00 Data variables: tasmin (time, lat, lon) float32 dask.array<chunksize=(512, 32, 32), meta=np.ndarray> tasmax (time, lat, lon) float32 dask.array<chunksize=(512, 32, 32), meta=np.ndarray> pr (time, lat, lon) float32 dask.array<chunksize=(512, 32, 32), meta=np.ndarray> - Attributes: (12/27) ? ^ + Attributes: (12/26) ? ^ Conventions: CF-1.5 title: Ouranos standard ensemble of bias-adjusted cl... - history: CMIP5 compliant file produced from raw ACCESS... + history: DATE_TIME_TZ CMOR rewrote data to com... institution: Ouranos Consortium on Regional Climatology an... - source: ACCESS1-3 2011. Atmosphere: AGCM v1.0 (N96 gr... - driving_model: ACCESS1-3 + source: NorESM1-M 2011 atmosphere: CAM-Oslo (CAM4-Os... + driving_experiment: historical,rcp85 ... ... + frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... - driving_institution: Commonwealth Scientific and Industrial Resear... + driving_institution: Norwegian Climate Centre - driving_institute_id: CSIRO-BOM ? ^^^^^^^^^ + driving_institute_id: NCC ? + ^ - driving_model_id: ACCESS1-3 _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 1 _ Notebook cell execution failed Cell 1: Cell outputs differ Input: from clisops.core import subset lon = [-75.4, -85, -65.5] # Longitude lat = [46.67, 41, 55.3] # Latitude ds_gridpoint = subset.subset_gridpoint(ds, lon=lon, lat=lat) display(ds_gridpoint) # Plot first year of tasmax data a = ds_gridpoint.tasmax.isel(time=slice(0, 365)).plot.line(x="time", figsize=(10, 4)) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data... ACCESS1-3' Skipping 39 identical leading characters in diff, use -v to show - time: 55152) ? - + time: 55115) ? + Coordinates: lat (site) float32 46.71 41.04 55.29 lon (site) float32 -75.38 -84.96 -65.46 - * time (time) datetime64[ns]/DATE/1950-01-02 ... 2100-12-31 + * time (time) object/DATE/TIME/.../DATE/00:00:00 Dimensions without coordinates: site Data variables: tasmin (time, site) float32 dask.array<chunksize=(512, 3), meta=np.ndarray> tasmax (time, site) float32 dask.array<chunksize=(512, 3), meta=np.ndarray> pr (time, site) float32 dask.array<chunksize=(512, 3), meta=np.ndarray> - Attributes: (12/27) ? ^ + Attributes: (12/26) ? ^ Conventions: CF-1.5 title: Ouranos standard ensemble of bias-adjusted cl... - history: CMIP5 compliant file produced from raw ACCESS... + history: DATE_TIME_TZ CMOR rewrote data to com... institution: Ouranos Consortium on Regional Climatology an... - source: ACCESS1-3 2011. Atmosphere: AGCM v1.0 (N96 gr... - driving_model: ACCESS1-3 + source: NorESM1-M 2011 atmosphere: CAM-Oslo (CAM4-Os... + driving_experiment: historical,rcp85 ... ... + frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... - driving_institution: Commonwealth Scientific and Industrial Resear... + driving_institution: Norwegian Climate Centre - driving_institute_id: CSIRO-BOM ? ^^^^^^^^^ + driving_institute_id: NCC ? + ^ - driving_model_id: ACCESS1-3 _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 3 _ Notebook cell execution failed Cell 3: Cell outputs differ Input: # Specify the longitude and latitude boundaries lon_bnds = [-80.5, -60.2] lat_bnds = [44, 55] ds1 = subset.subset_bbox(ds, lon_bnds=lon_bnds, lat_bnds=lat_bnds) display(ds1) # Plot a map of first timestep a = ds1.tasmax.isel(time=0).plot() Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data...id: NCC' == '<xarray.Data... ACCESS1-3' Skipping 50 identical leading characters in diff, use -v to show - time: 55152) ? - + time: 55115) ? + Coordinates: * lat (lat) float32 54.96 54.87 54.79 54.71 ... 44.29 44.21 44.12 44.04 * lon (lon) float32 -80.46 -80.38 -80.3 -80.21 ... -60.38 -60.3 -60.21 - * time (time) datetime64[ns]/DATE/1950-01-02 ... 2100-12-31 + * time (time) object/DATE/TIME/.../DATE/00:00:00 Data variables: tasmin (time, lat, lon) float32 dask.array<chunksize=(512, 20, 28), meta=np.ndarray> tasmax (time, lat, lon) float32 dask.array<chunksize=(512, 20, 28), meta=np.ndarray> pr (time, lat, lon) float32 dask.array<chunksize=(512, 20, 28), meta=np.ndarray> - Attributes: (12/27) ? ^ + Attributes: (12/26) ? ^ Conventions: CF-1.5 title: Ouranos standard ensemble of bias-adjusted cl... - history: CMIP5 compliant file produced from raw ACCESS... + history: DATE_TIME_TZ CMOR rewrote data to com... institution: Ouranos Consortium on Regional Climatology an... - source: ACCESS1-3 2011. Atmosphere: AGCM v1.0 (N96 gr... - driving_model: ACCESS1-3 + source: NorESM1-M 2011 atmosphere: CAM-Oslo (CAM4-Os... + driving_experiment: historical,rcp85 ... ... + frequency: day modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... - driving_institution: Commonwealth Scientific and Industrial Resear... + driving_institution: Norwegian Climate Centre - driving_institute_id: CSIRO-BOM ? ^^^^^^^^^ + driving_institute_id: NCC ? + ^ - driving_model_id: ACCESS1-3 _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb::Cell 6 _ Notebook cell execution failed Cell 6: Cell outputs differ Input: # Align data to a starting year ds_sub = subset.subset_time(ds, start_date="1981") print( f"Subset time using start_date only\nstart: {ds_sub.time.min().values};\tend: {ds_sub.time.max().values}\n" ) # Select a temporal slice ds_sub = subset.subset_time(ds, start_date="1981-08-05", end_date="2084-06-15") print( f"Subset time using both start_date & end_date\nstart: {ds_sub.time.min().values};\tend: {ds_sub.time.max().values}" ) Traceback: mismatch 'stdout' assert reference_output == test_output failed: 'Subset time ...TE/00:00:00\n' == 'Subset time ...0.000000000\n' Subset time using start_date only - start: 1981-01-01T00:00:00.000000000; end: 2100-12-31T00:00:00.000000000 + start:/DATE/00:00:00; end:/DATE/00:00:00 Subset time using both start_date & end_date - start: 1981-08-05T00:00:00.000000000; end: 2084-06-15T00:00:00.000000000 + start:/DATE/00:00:00; end:/DATE/00:00:00 _ PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb::Cell 0 _ Notebook cell execution failed Cell 0: Cell outputs differ Input: import os os.environ["USE_PYGEOS"] = "0" # force use Shapely with GeoPandas import warnings import numba warnings.simplefilter("ignore", category=numba.core.errors.NumbaDeprecationWarning) import geopandas as gpd import matplotlib.pyplot as plt import xarray as xr from clisops.core import subset from dask.diagnostics import ProgressBar from siphon.catalog import TDSCatalog from xclim import atmos warnings.simplefilter("ignore") # TODO change address url = "https://boreas.ouranos.ca/twitcher/ows/proxy/thredds/catalog/datasets/simulations/bias_adjusted/cmip5/ouranos/cb-oura-1.0/catalog.xml" # TEST_USE_PROD_DATA # Create Catalog cat = TDSCatalog(url) # Subset over the Gasp�� peninsula in eastern Quebec gaspe = gpd.GeoDataFrame.from_file( "/notebook_dir/pavics-homepage/tutorial_data/gaspesie_mrc.geojson" ) ds = subset.subset_shape( xr.open_dataset(cat.datasets[0].access_urls["OPENDAP"], chunks=dict(time=256)), shape=gpd.GeoDataFrame(geometry=gaspe.buffer(0.05)), ) # What we see here is only a representation of the full content, the entire data set hasn't been loaded. display(ds) # plot of single day tasmin a = ds.tasmin.isel(time=0).plot(figsize=(10, 4)) Traceback: mismatch 'text/plain' assert reference_output == test_output failed: '<xarray.Data... EPSG:4326' == '<xarray.Data... EPSG:4326' Skipping 48 identical leading characters in diff, use -v to show - time: 55152) ? - + time: 55115) ? + Coordinates: * lat (lat) float32 49.29 49.21 49.12 49.04 ... 48.12 48.04 47.96 47.87 * lon (lon) float32 -67.71 -67.63 -67.55 -67.46 ... -64.3 -64.21 -64.13 - * time (time) datetime64[ns]/DATE/1950-01-02 ... 2100-12-31 + * time (time) object/DATE/TIME/.../DATE/00:00:00 Data variables: tasmin (time, lat, lon) float32 dask.array<chunksize=(256, 18, 44), meta=np.ndarray> tasmax (time, lat, lon) float32 dask.array<chunksize=(256, 18, 44), meta=np.ndarray> pr (time, lat, lon) float32 dask.array<chunksize=(256, 18, 44), meta=np.ndarray> crs int64 1 - Attributes: (12/28) ? ^ + Attributes: (12/27) ? ^ Conventions: CF-1.5 title: Ouranos standard ensemble of bias-adjusted cl... - history: CMIP5 compliant file produced from raw ACCESS... + history: DATE_TIME_TZ CMOR rewrote data to com... institution: Ouranos Consortium on Regional Climatology an... - source: ACCESS1-3 2011. Atmosphere: AGCM v1.0 (N96 gr... - driving_model: ACCESS1-3 + source: NorESM1-M 2011 atmosphere: CAM-Oslo (CAM4-Os... + driving_experiment: historical,rcp85 ... ... + modeling_realm: atmos target_dataset: CANADA : ANUSPLIN interpolated Canada daily 3... target_dataset_references: CANADA : https://doi.org/10.1175/2011BAMS3132... - driving_institution: Commonwealth Scientific and Industrial Resear... + driving_institution: Norwegian Climate Centre - driving_institute_id: CSIRO-BOM ? ^^^^^^^^ + driving_institute_id: NCC ? + ^ - driving_model_id: ACCESS1-3 crs: EPSG:4326
Check out this pull request on
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
Most of the graphs are different, weird.
Only merge if https://github.com/bird-house/birdhouse-deploy/pull/413 can be merged.
This is mostly to have a nice notebook diff.