Ouranosinc / PAVICS-landing

The landing page serving as the entrance to PAVICS
1 stars 0 forks source link

New pavics-hydro tutorial based on RavenPy #80

Closed huard closed 1 month ago

huard commented 3 months ago

Matching Jenkins changes to test these new notebooks by default during nightly builds: https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/139

review-notebook-app[bot] commented 3 months ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

tlvu commented 2 months ago

Jenkins do not see these new notebooks yet. I'll have to update Jenkins config before we can start testing those new notebooks automatically with Jenkins.

tlvu commented 2 months ago

Tried this on Jenkins, both for the current Stable image and the latest build in the Gamma image (with latest xclim, ravenpy) and both have this error:

23:39:18  _ PAVICS-landing-hydro/content/notebooks/hydrology/03_Watershed_properties.ipynb::Cell 10 _
23:39:18  Notebook cell execution failed
23:39:18  Cell 10: Cell execution caused an exception
23:39:18  
23:39:18  Input:
23:39:18  properties, dem = terrain_resp.get(asobj=True)
23:39:18  dem = dem.band_data
23:39:18  elevation = properties[0]["elevation"]
23:39:18  slope = properties[0]["slope"]
23:39:18  aspect = properties[0]["aspect"]
23:39:18  
23:39:18  terrain = {"elevation": elevation, "slope": slope, "aspect": aspect}
23:39:18  display(terrain)
23:39:18  
23:39:18  Traceback:
23:39:18  
23:39:18  ---------------------------------------------------------------------------
23:39:18  AttributeError                            Traceback (most recent call last)
23:39:18  Cell In[1], line 2
23:39:18        1 properties, dem = terrain_resp.get(asobj=True)
23:39:18  ----> 2 dem = dem.band_data
23:39:18        3 elevation = properties[0]["elevation"]
23:39:18        4 slope = properties[0]["slope"]
23:39:18  
23:39:18  File /opt/conda/envs/birdy/lib/python3.11/site-packages/xarray/core/common.py:285, in AttrAccessMixin.__getattr__(self, name)
23:39:18      283         with suppress(KeyError):
23:39:18      284             return source[name]
23:39:18  --> 285 raise AttributeError(
23:39:18      286     f"{type(self).__name__!r} object has no attribute {name!r}"
23:39:18      287 )
23:39:18  
23:39:18  AttributeError: 'DataArray' object has no attribute 'band_data'
huard commented 1 month ago

Now ?

tlvu commented 1 month ago

'band_data' error is gone :D Only have output difference. I can take care to refresh the output.

tlvu commented 1 month ago

This PR is almost ready to be merged, except one weird issue: Jenkins against other hosts than the production host generate an output diff that seems to be different in precision only ! 0264687799 vs 02646877995, it's only the extra '5' at the end !

Both machines have the same RavenWPS version (pavics/raven:0.18.1) and all point to the same GeoServer on production host.

@huard @tlogan2000 I could use NBVAL_IGNORE_OUTPUT again but this diff just make no sense. Any other ideas?

18:17:49  _ PAVICS-landing-hydro/content/notebooks/hydrology/03_Watershed_properties.ipynb::Cell 10 _
18:17:49  Notebook cell execution failed
18:17:49  Cell 10: Cell outputs differ
18:17:49  
18:17:49  Input:
18:17:49  properties, dem = terrain_resp.get(asobj=True)
18:17:49  elevation = properties[0]["elevation"]
18:17:49  slope = properties[0]["slope"]
18:17:49  aspect = properties[0]["aspect"]
18:17:49  
18:17:49  terrain = {"elevation": elevation, "slope": slope, "aspect": aspect}
18:17:49  display(terrain)
18:17:49  
18:17:49  Traceback:
18:17:49   mismatch 'text/plain'
18:17:49  
18:17:49   assert reference_output == test_output failed:
18:17:49  
18:17:49    "{'elevation'...402646877995}" == "{'elevation'...940264687799}"
18:17:49    
18:17:49    Skipping 82 identical leading characters in diff, use -v to show
18:17:49    - 0264687799}
18:17:49    + 02646877995}
18:17:49    ?           +
18:17:49  
18:17:49  
18:17:49  _ PAVICS-landing-hydro/content/notebooks/hydrology/03_Watershed_properties.ipynb::Cell 13 _
18:17:49  Notebook cell execution failed
18:17:49  Cell 13: Cell outputs differ
18:17:49  
18:17:49  Input:
18:17:49  all_properties = {**shape_info, **land_use, **terrain}
18:17:49  display(all_properties)
18:17:49  
18:17:49  Traceback:
18:17:49   mismatch 'text/plain'
18:17:49  
18:17:49   assert reference_output == test_output failed:
18:17:49  
18:17:49    "{'area': 55....402646877995}" == "{'area': 55....940264687799}"
18:17:49    
18:17:49    Skipping 501 identical leading characters in diff, use -v to show
18:17:49    - 0264687799}
18:17:49    + 02646877995}
18:17:49    ?           +
huard commented 1 month ago

Don't know, and not worth fussing over.

tlogan2000 commented 1 month ago

Don't know, and not worth fussing over.

I agree. I think if the cell executes with success then we can ignore theses differences even if they are a bit strange (same version etc etc)

tlvu commented 1 month ago

Don't know, and not worth fussing over.

I agree. I think if the cell executes with success then we can ignore theses differences even if they are a bit strange (same version etc etc)

I agree, I was just wondering if the difference indicate potentially some sort of problem in the code, ex: rounding error. Will add NBVAL_IGNORE_OUTPUT.

tlvu commented 1 month ago

Jenkins all passed, merge when ready.

tlvu commented 1 month ago

Sorry, have to merge this one so I can merge https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/pull/139, which will fix https://github.com/Ouranosinc/PAVICS-landing/issues/92.