EC view changes from demo

[wdwatkins]

Tableau things:

• [x] Add dropdown to geographic view for other state metrics (parquets "state_week_vs_year" and state_traffic_population_percentages")
• [x] List which apps are contained in 10% up/down etc categories
• [x] Visually separate 'Trends summary' box a bit more; maybe it should go below the bar chart, since that chart goes with the higher up dropdowns?
• [x] Change trends view to % change from previous year
• [x] App milestone/event selections affect the geographic view—is this avoidable?
• [ ] Add toggle for Flagship/IOW to milestones/event annotations. Will need to wait until we have some more events to do this
• [x] Move trends and event annotations view to the top
• [ ] Nice to have: ability to apply event to multiple apps; click [wait on this; easier to do with app groups marked in an Athena table?]
• [ ] Nice to have: click on event in view, isolate data to the corresponding application
• [x] add latest date of data on EC dashboard

Sharepoint things:

• [x] Character limit for event names? 85 characters, i set a limit in the form.
• [x] Add description field, appears in hover in Tableau Description field was created and displays on hover.

Data things:

• [ ] Create an all-NWIS web category? Think about the level of overlap between existing breaks
• [x] Explore statistics for describing trends in web traffic used by other organizations
• [x] For dropdown choices that don't correspond to calendar months, etc, change to 7, 30, 365 days

[mhines-usgs]

on each of the checklist items/points for our discussion at 1230ct tableau things list: 1) no big deal, just waiting for data. not entirely sure if i can squeeze it into the same dropdown period but i have an idea and will try that before needing to check in again on that. just waiting for corrected data to show up in s3. (pipeline job seems to be failing) 2) done - now present in a hover over the symbol and the bin/class 3) not a big deal, just waiting to make other changes/additions before moving things around since the layout is the last part i would want to do to make sure i don't have to do it again. dashboard layout design is a bit clunky. 4) done - present on all dashboards, changed the yaxis label to match 5) i may just have to disable highlighting, or when i create the new dashboard with things moved around, need to disable all default 'actions' and re-implement specific actions i want. tableau tries to be very helpful but often you spend time battling the automatic things. 6) this is not possible until we actually have events for each application. there is nothing to make application name sets with until we have data we can use in a filter. so i suggest we shelve this. 7) no big deal, will do when I recreate dashboard after making suggested modifications. bundle with 3 above in my opinion. 8) not sure this is possible in tableau alone, might need to modify the data so that there are obvious flags to draw from first. this would also help with all the sets i've had to create/maintain in tableau. might be nice to push this forward to next year and address more holistically throughout the dashboard. 9) have not attempted this yet, not sure if it's possible but seems reasonable.

sharepoint things 1) is this requesting the limit or suggesting we impose a limit? we can impose a limit in sharepoint, right now it is set to 255 characters, but i have not done testing to see what is too much. can do. 2) yes, we can add a description field to show on hover, can do.

[wdwatkins]

@wdwatkins needs to fix the duplicate REGION field in the population percentage file

[wdwatkins]

One thing I forgot to add here—I think adding the date of the latest data came up during the demo. Is that an easy thing to do?

[mhines-usgs]

yeah i think so, basically i could look at one of the files that has the daily session data and just grab the max date?

[wdwatkins]

That should do it

[mhines-usgs]

manually updated the state_week_vs_year.parquet for now in s3, the job seems to fail often with the time out issue. i updated the ec dashboard on both tableau server instances but with the data changes i have to go repair a few things on the old views and copy over some of the changes i made on the ec dashboard over to the 'more detailed' dashboard.

[wdwatkins]

Yeah, I think adding anti-sample calls substantially increased the amount of data being pull from NWIS web. There is an option to cache calls on disk that I think can be added fairly easily that I want to try. That should eliminate duplicate API calls that happen when we rerun the job on the same day, and in some cases when the anti-sample method makes the same calls day after day since it has to retrieve data in small chunks.

I also have a PR in with the google analytics package that once resolved should let us use a 'slow fetch' option that should help with the 429 errors.

[mhines-usgs]

corresponding edits made to the more detailed views and pushed up to both tableau servers. going to do some documentation but if you have time to skim over the changes and if you have feedback or want to talk through them sometime tomorrow, feel free to throw a calendar event my way?

[wdwatkins]

Sounds good. I'll go through the 'metadata' page too, I think we may want to add to that for the EC view. Or otherwise have some text for Nate et al to refer too.

[mhines-usgs]

perfect, thank you!

[mhines-usgs]

specific thoughts on feedback

• language OK for date of last update?
• left the legend off the geographic map views, but can add back if think it's needed.
• geographic lists currently do not update either with different views selected, but if that would be desired, i can address that, wasn't sure how useful that info is?
• defaults ok?
• to fill white space around the tiny trends plot, made it bigger and included a dynamic list, that updates on hover or just lists all the apps in that grouping when not hovering
• left highlight active on the trends views so that they could be interactive with eachother, but does seem to affect other parts of the dashboard still. i can disable the highlighting altogether if that's confusing

[mhines-usgs]

• [x] add back all months selection on trends view plots

• [x] all trends use PREVIOUS duh

• [x] week over year percent change - make a diverging color ramp from 0 from week over year

• [x] text about the geographic view that explains the views more

• [x] on geo views, extend the state/country lists to update too for other choices (not just visits)

• [x] population ratios - same thing diverge from 1 (below 1 is underrepresented, above 1 overrepresented)

• [x] disable highlighting on trends views

• [x] replicate geo view changes on detailed dashboard

• [x] push changes to tableau.usgs.gov for both dashboards

• [ ] (needs data change first) adding data for nwis as a whole that will need to be added to the flagship group

[mhines-usgs]

the checked off changes for the EC Dashboard are up on tableau.chs.usgs.gov - for some reason I'm unable to change the connection details at the moment (getting an error message) and push it to the tableau.usgs.gov server with the workbook open from the internal server. i'll try again tomorrow before reaching out to them for help :( that's the way i usually 'deploy' changes to both servers after making changes

[mhines-usgs]

was able to push this up this morning. i'm going to do another task for gcmrc quick this morning before addressing the correlated changes needed in the other workbook, and adding to the documentation about how to edit data source paths and republish changes to both servers.

[wdwatkins]

Can you change the flagship apps trends Y axis to "Year-over-year percent change in visits"? And is it easy to make the actual numbers appear in the heat-table hover, in addition to the percent changes?

[mhines-usgs]

Yes to both. Assuming we want to do the same for all of those (IoW too?) across both workbooks?

[wdwatkins]

Some explanatory text for the geographic views:

Visits: Total visits per state during the specified time period. Tends to be affected by state population. Traffic share to population share ratio: The ratio of a state's share of visits over a time period and the state's share of population in 2019. Negative values indicate a state is underrepresented in visits compared to its population, positive numbers indicate a state is overrepresented compared to population. Previous 7 days vs average: Percent difference between the total visits per state over the last 7 days and the average 7-day total traffic for the last 365 days. Attempts to capture spikes in traffic from individual states. Negative numbers indicate below average (for the past 365 days) traffic, positive numbers above average. Does not account for seasonal patterns. This metric does not use the "Period" dropdown.

Can you make the dropdown go in this order too? And change the dropdown options to the titles here?

[mhines-usgs]

pushed above changes for explanatory text and names of views in the parameter selection to tableau.chs.usgs.gov but again hitting that error after not getting it earlier this morning on the tableau.usgs.gov server when I attempt to change to it. will try restarting before going to chs to get help/complain...

[mhines-usgs]

figured out the connection issue and all the checked off changes are now up on both servers