search

USGS-CMG / usgs-cmg-portal

do_convert.sh
6 stars 13 forks source link

flow directions and their representations #295

Open emontgomery-usgs opened 5 years ago

emontgomery-usgs commented 5 years ago

@kwilcox

Rich's answer to the question about how to display the wind, wave, current direction variables in the portal is "as they are".

This means than wind and waves should be "from_direction" and current (CD_310) should be "to_direction". In files where CD_310 already exists, it will be correct ("to") but where you compute it, I believe it will be "from", but should be "to".

I looked at FI14-989 (https://cmgdata.usgsportals.net/#module-metadata/590b5a22-b61e-11e4-b2db-00265529168c/6878f136-b61e-11e4-a8f5-00265529168c) and it looks like the portal only displays speed for currents (no direction), so the fact that the direction is likely to be 180 off what's expected isn't exposed.

rsignell-usgs commented 5 years ago

@emontgomery-usgs , looking into this.

So we have established that:

FWIW, I checked the standard_name list and both sea_water_from_direction and sea_water_to_direction are listed as options. Perhaps they are new?

I checked a sample location in West Falmouth and the current looks okay (90 degrees is current toward east): http://erddap.axds.co/erddap/tabledap/56455.graph?eastward_sea_water_velocity_8871aqd_a%2Csea_water_velocity_to_direction_8871aqd_a%2C&time%3E=2010-09-02T00%3A00%3A00Z&time%3C=2010-09-09T00%3A00%3A00Z&.draw=markers&.marker=5%7C5&.color=0x000000&.colorBar=%7C%7C%7C%7C%7C&.bgColor=0xffccccff

@kwilcox, one small thing to fix: we have standard_name=sea_water_velocity_to_direction but that is not a valid standard_name. It should be standard_name=sea_water_to_direction.

emontgomery-usgs commented 5 years ago

@rsignell-usgs @kwilcox The root of the problem is applying normalize_vectors() to all direction variables in the process of generating the CF version. I disabled this function in the _noregex version to allow the CF directions to retain the same from/to that they have in EPIC. Not sure what this will do downstream if ingested to the portal- it assumes all direction data points the same way, so I think we need some changes to allow wind and wave directions to be displayed as "from" and currents to be displayed as "to".

rsignell-usgs commented 5 years ago

@emontgomery-usgs can you please point to an example in the portal where something is going in the wrong direction?

emontgomery-usgs commented 5 years ago

@rsignell-usgs Sure- the issue is in "direction" variables not east and north. example to follow.

I am perplexed about what your ERDDAP example shows. When I compare the east data in the actual source files, the only difference is a factor of 100 to convert units, and when I plot u_epic vs u_cf, I get nice 1:1 line, not the wiggles your ERDDAP plot shows... I am not concerned about the CF and portal east and north, the problem is how the (mostly computed for CF) direction data is oriented.

emontgomery-usgs commented 5 years ago

@rsignell-usgs @kwilcox - maybe this is partially solved in the new portal, since the direction data is not shown. at https://cmgdata.usgsportals.net/#metadata/56459/station, what is displayed under "currents" is just speed, and I didn't find direction displayed.

But if you delve a bit deeper looking for which sensor the data came from, you get to https://cmgdata.usgsportals.net/#metadata/56459/station/15/sensor, and the direction rose is displayed as a "to" direction. This is current data from a TRDI workhorse ADCP, so "to" direction is correct. The source data is http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/CF-1.6/FI14/9981wh-a.nc.html, and in CD_310, there's standard_name: direction_of_sea_water_velocity, which should be changed to "sea_water_to_direction"

We don't ingest wave data now, but in testing the _regex version of collect.py, I ran into the problems with what happens when waves data is converted to CF. From from tests using the same instrument (9881) the CF version I generate has wave_direction also converted to a "to" direction. http://geoport.whoi.edu/thredds/dodsC/silt/usgs/Projects/stellwagen/Data_tests/tst_wvsa/10631whVp-cal_z2.nc.html. In the wvdir variable there are these attributes, and the data is rotated 180: standard_name: sea_surface_wave_to_direction NOTE: Direction FROM which waves are propagating, measured clockwise from north Not what we want for waves (or wind data, but I haven't looked at that).

The example is not in the portal now, but will be if when we ingest the waves files. I think we need to remove or modify normalize_vectors() prior to ingesting waves data, and make sure the portal won't try to display wave directions as "to" directions.

emontgomery-usgs commented 5 years ago

I found an example from the Met Buoy in the same experiment- it correctly has wind speed displayed as "from".
https://cmgdata.usgsportals.net/#metadata/56422/station

emontgomery-usgs commented 5 years ago

If you want to try to ingest sample waves files of several types, the files here are the outputs of my most recent run of collect_noregex.py with normalize_vectors() disabled. It would be interesting to see how these display! https://gamone.whoi.edu/thredds/silt/usgs/Projects/stellwagen/Data_tests/tst_wvsb/catalog.html thanks!

rsignell-usgs commented 5 years ago

TL;DR, nothing that needs to be done at this point unless we want to experiment with wave data?

emontgomery-usgs commented 5 years ago

@rsignell-usgs @kwilcox, I made the standard_name for CD_310 to sea_water_to_direction (mentioned earlier in the thread) in my local version of epic2cf.

rsignell-usgs commented 5 years ago

@emontgomery-usgs, okay. Is there any action needed on this by @kwilcox ?

emontgomery-usgs commented 5 years ago

@rsignell-usgs, In order to move forward using the correct standard names, my version of epic2cf needs to be pulled into the axiom version. The way I got set-up does not allow for a pull request.

kwilcox commented 5 years ago

@emontgomery-usgs Now that epic2cf is updated we can move on with the bullets!