search

Ouranosinc / PAVICS-e2e-workflow-tests

Test user-level workflow.
Apache License 2.0
0 stars 2 forks source link

docker: new build for new XClim and to get Dask dashboard and Panel server app to work #105

Closed tlvu closed 2 years ago

tlvu commented 2 years ago

Overview

Still have to pin dask for performance and shapely for notebook error.

Dask dashboard and Panel server app work both by manually mapping http://localhost:<PORT> to https://pavics.ouranos.ca/jupyter/user/<USER>/proxy/<PORT> as seen in screenshots below:

Screenshot from 2022-04-12 15-20-58

Screenshot from 2022-04-12 16-29-06

The "Render with Panel" green button also works and avoid having to deal with manual URL rewrite: Screenshot from 2022-05-04 15-18-03

Relevant Changes

# new
>   - dask-labextension=5.2.0=pyhd8ed1ab_0
>   - jupyter-panel-proxy=0.2.0a2=py_0
>   - jupyter-server-proxy=3.2.1=pyhd8ed1ab_0

# removed, interfere with panel
<     - handcalcs==1.4.1

<   - xclim=0.34.0=pyhd8ed1ab_0
>   - xclim=0.36.0=pyhd8ed1ab_0

<   - cf_xarray=0.6.3=pyhd8ed1ab_0
>   - cf_xarray=0.7.2=pyhd8ed1ab_0

<   - clisops=0.8.0=pyh6c4a22f_0
>   - clisops=0.9.0=pyh6c4a22f_0

# downgrade by clisops
<   - pandas=1.4.1=py38h43a58ef_0
>   - pandas=1.3.5=py38h43a58ef_0

<   - rioxarray=0.10.3=pyhd8ed1ab_0
>   - rioxarray=0.11.1=pyhd8ed1ab_0

<   - nc-time-axis=1.4.0=pyhd8ed1ab_0
>   - nc-time-axis=1.4.1=pyhd8ed1ab_0

<   - roocs-utils=0.5.0=pyh6c4a22f_0
>   - roocs-utils=0.6.1=pyh6c4a22f_0

<   - panel=0.12.7=pyhd8ed1ab_0
>   - panel=0.13.1a2=py_0

<   - plotly=5.6.0=pyhd8ed1ab_0
>   - plotly=5.7.0=pyhd8ed1ab_0

Related Issue / Discussion

Test

Additional Information

tlvu commented 2 years ago

@tlogan2000 @aulemahal This is a newer "beta" build than the one you tried last week to pull latest changes of all packages. You'll have to stop and restart your Jupyter server.

@tlogan2000 Please perform any extra manual Bokeh performance test. I've manually tested the homepage notebook 5 and no apparent performance regression observed.

Zeitsperre commented 2 years ago

Not sure if it matters, but pandas in clisops is pinned below v1.4 right now. I'm not sure if that might cause environment inconsistencies. https://github.com/roocs/clisops/blame/d7a339addf436af0c57a4f2b3e305077ce07fa11/requirements.txt#L3

tlvu commented 2 years ago

Not sure if it matters, but pandas in clisops is pinned below v1.4 right now. I'm not sure if that might cause environment inconsistencies. https://github.com/roocs/clisops/blame/d7a339addf436af0c57a4f2b3e305077ce07fa11/requirements.txt#L3

This is weird, we did not explicitly specify pandas (https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/dc6745158f589360573ac7f48db1f207fe30e5ea/docker/environment.yml#L50) so that version should have been "negotiated" between all the indirect dependencies.

tlvu commented 2 years ago

Not sure if it matters, but pandas in clisops is pinned below v1.4 right now. I'm not sure if that might cause environment inconsistencies. https://github.com/roocs/clisops/blame/d7a339addf436af0c57a4f2b3e305077ce07fa11/requirements.txt#L3

This is weird, we did not explicitly specify pandas (

https://github.com/Ouranosinc/PAVICS-e2e-workflow-tests/blob/dc6745158f589360573ac7f48db1f207fe30e5ea/docker/environment.yml#L50 ) so that version should have been "negotiated" between all the indirect dependencies.

Found it.

Clisops introduced pandas pin since 0.9.0 only, we have 0.8.0 https://github.com/conda-forge/clisops-feedstock/blob/2bdbb0ef9c1297d8dbb3483dcc4664dbe84ec204/recipe/meta.yaml#L26

I guess I have to force clisops >= 0.9.0 to get it. Somehow the build resolves to an older clisops. mamba is faster than conda but sometimes it is not precise :(

Thanks for the catch.

review-notebook-app[bot] commented 2 years ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB