GTS notebook

[ocefpaf]

Closes #351

nbviewer preview: https://nbviewer.jupyter.org/github/ocefpaf/notebooks_demos/blob/GTS/notebooks/2020-10-10-GTS.ipynb

• [ ] create a map with the buoys and pop-up with the info
• [ ] find the buoy position from NDBC
• [ ] hexmap/heatmap with the number of obs

[review-notebook-app[bot]]

Check out this pull request on 

Review Jupyter notebook visual diffs & provide feedback on notebooks.

---

Powered by ReviewNB

[MathewBiddle]

I pulled this notebook over locally and played around with it. I was able to find an NDBC resource for station locations, match those up with the locationID and make a map with folium. Unfortunately, adding a popup seems to be problematic.

[ocefpaf]

I pulled this notebook over locally and played around with it. I was able to find an NDBC resource for station locations, match those up with the locationID and make a map with folium. Unfortunately, adding a popup seems to be problematic.

Those for pointing me to that xml. I created a function to get the positions and crated a map with a HTML summary table as a popup. BTW, did you get an error with the popup? Let me know if I can help.

[MathewBiddle]

Looks good! I think the popup issue was related to Chrome, as documented here https://github.com/python-visualization/folium/issues/812. Running on Firefox, I don't have any issues.

I added this snippet to colorize the icons based on if there are observations for both met and wave (green), just met or just wave (orange), or neither (red). See below:

import folium
from folium.plugins import Fullscreen

map = folium.Map(location=[df['lat'].mean(), df['lon'].mean()], zoom_start=2)
Fullscreen().add_to(map)

for index, row in df.iterrows():
    if row['met'] == 0 | row['wave'] == 0:
        color = 'red'
    elif row['met'] == 0:
        color = 'orange'
    elif row['wave'] == 0:
        color = 'orange'
    else:
        color = 'green'

    folium.Circle(
        location=[row['lat'],row['lon']],
        radius = 5,
        color = color,
        #fill = True,
        #alpha = 0.5,
        #icon = folium.Icon(color=color,
        #                  icon= 'list-alt',
        #                  ),
        popup = 'Station:%s\nMet:%.0f\nWave:%.0f'% (row['locationID'],row['met'],row['wave'])
    ).add_to(map)

map

[ocefpaf]

Looks good! I think the popup issue was related to Chrome, as documented here python-visualization/folium#812. Running on Firefox, I don't have any issues.

I'm using chrome but maybe the issue is the iopub size. I always start my notebook server with --NotebookApp.iopub_data_rate_limit=1000000000.

I added this snippet to colorize the icons based on if there are observations for both met and wave (green), just met or just wave (orange), or neither (red).

Nice! Let's add that here.

[MathewBiddle]

This looks good to me. Not sure where to go from here.

[ocefpaf]

This looks good to me. Not sure where to go from here.

I need to create a narrative for this to be a notebook on the data demo center. I'll work on that this week and ping you for a final review.

My understanding is that @kbailey-noaa also wants a version of this in a live doc page. That may be more challenging. We can touch base on that in our next meeting.

[MathewBiddle]

Any updates? I think the notebook will suffice for now, but we should explore how to embed this as a live figure somewhere.

[kbailey-noaa]

Minor changes: For each of the bar graphs can you label the y-axis "# observations" and allow for the full number to be written out (rather than the 1E6 notation)? And then can you also include the actual # per month somewhere? Whether a table below the graph or in the graph itself - doesn't matter. A table might be easier to digest though. Becca (in IOOS) needs a way to tally the # obs for a metrics report and said as long as she knows # per month she'd have all the info she needs. I don't think she could easily glean that from the bar graph as is.

[ocefpaf]

Any updates? I think the notebook will suffice for now, but we should explore how to embed this as a live figure somewhere.

For some reason I "unsubscribed" from updates here and missed the updates. I probably cliked on it by mistake. (Still getting used to GH new notification interface.)

I incorporated @kbailey-noaa suggestions in the next commit.

[kbailey-noaa]

@ocefpaf I don't know where to go to look at the latest version :( The nbviewer preview link gave me a 404 error.

[ocefpaf]

Just updated the link above.

[MathewBiddle]

I'm curious why you set the ERDDAP constraint to >= January, 1, 2019?

Maybe we could just constrain to the last year of data?

e.constraints = {

  "time>=": (date.today()-timedelta(days=365)).strftime('%Y-%m-%dT%H:%M:%SZ'),

}

[ocefpaf]

I'm curious why you set the ERDDAP constraint to >= January, 1, 2019?

That is the first data point if I'm not mistaken.

Maybe we could just constrain to the last year of data?

Sure. That makes sense. Originally we were trying to get all the data as far back as possible but, in this case, it will be the same.

[ocefpaf]

but, in this case, it will be the same.

Well, not really if we use today as a base. Do we want all data as far back as the first point? Only 2019? Only 2020? Or today-1year?

[ocefpaf]

Well, not really if we use today as a base. Do we want all data as far back as the first point? Only 2019? Only 2020? Or today-1year?

As we discussed in the meeting I'm starting from Sep-2019 up to Oct-2020, but the server only has data up to Jun-2020. This should be ready for a next round of reviews.

[MathewBiddle]

Can we add a legend to the map? red = no met, orange = no wave, green = obs for both.

Otherwise, it looks good to me.

[ocefpaf]

Can we add a legend to the map? red = no met, orange = no wave, green = obs for both.

Sadly legend on folium maps are still not easy to do. ipyleafet makes it easier but then the map is not published without a live kernel. I could try holoviz/bokeh but at this point I guess that the shortest path is to add some text above it with that information. Is that OK?

[MathewBiddle]

That's fine.

[ocefpaf]

That's fine.

Yep, as I thought, we cannot get ipyleaflet to display on nbviewer: https://nbviewer.jupyter.org/github/ocefpaf/notebooks_demos/blob/GTS/notebooks/2020-10-10-GTS.ipynb

Pity, that map is a bit nicer and has the legend. I'll make a few more tests and then revert to plan B in https://github.com/ioos/notebooks_demos/pull/353#issuecomment-706354736

[ocefpaf]

@MathewBiddle latest commit worked locally to display the map as HTML. However, that did not work on nbviewer, probably some JS is missing. I'm not really sure how that will play out in our page. Do you want to try it? We can revert back and add a text only legend later if this does not work.

[MathewBiddle]

@ocefpaf, I couldn't get the latest commit to work locally. Cell runs fine, but doesn't display a map. I wonder if it's related to https://github.com/ioos/notebooks_demos/pull/353#issuecomment-675056979?

[ocefpaf]

@ocefpaf, I couldn't get the latest commit to work locally. Cell runs fine, but doesn't display a map. I wonder if it's related to #353 (comment)?

Nope. Not sure. Maybe your browser is blocking the ipyleaflet JS? Are you on Jupyter classic notebooks or JupyterLab?

[MathewBiddle]

Jupyter classic notebooks. Oddly enough, it seems to be displaying now... We can push and see what happens?

[ocefpaf]

Let's go for it!

[ocefpaf]

We can push and see what happens?

Page build is broken :-(

Looks like nbconvert changed how it register the templates. I'll work on this during DMAC breaks.

[kbailey-noaa]

@ocefpaf @MathewBiddle I got a request for GTS data flow for FY21 Q1 (oct 1- dec 31). I can't seem to find the notebook through this site anymore - I get the 404 error. Is this something you could fix soon?

[MathewBiddle]

Its in master now. https://github.com/ioos/notebooks_demos/blob/master/notebooks/2020-10-10-GTS.ipynb

[MathewBiddle]

@ocefpaf is this ready for the code gallery? If so. how do we push it there?

[ocefpaf]

@ocefpaf is this ready for the code gallery? If so. how do we push it there?

Travis-CI broke our publishing docs, let me finish the move to GHA and we'll get this on up. Working on it...