F-TDS and server-side subsampling

[karlmsmith]

Reported by @AnsleyManke on 10 Apr 2013 22:15 UTC We've demonstrated using an F-TDS script to populate an empty dataset with variables which come from a Discrete Sampling Geometry file, by sending a mask definition to an F-TDS script which defines variables with the mask applied, for a sampling of the trajectories.

The F-TDS process adds attributes to the variables for internal use, and now we're seeing those attributes exposed.

For instance, this un-encoded URL will pick out a couple of WMO ID's and defines all the original variables, sampled, in the file sample.nc

http://dunkel.pmel.noaa.gov:8280/thredds/dodsC/las/DrifterBuoyData_1_5000.nc_expr_{/home/data/ansley/sample.nc_cr_let mask=if (wmo EQ 32519 OR wmo EQ 52801) then 1}{go data_mask_script}.html

Encodes as,

http://dunkel.pmel.noaa.gov:8280/thredds/dodsC/las/DrifterBuoyData_1_5000.nc_expr_%7B/home/data/ansley/sample.nc_cr_let+mask%3Dif+%28wmo+EQ+32519+OR+wmo+EQ+52801%29+then+1%7D%7Bgo%20data_mask_script%7D.html

and the output shows 7 trajectories using those wmo id's, with a total of 14621 obs.

The ncdump header output of the above encoded url is attached. Things to look at:

The dataset name is "sample". This is the name of the container where the sampled variables are defined. It would be good to have the original dataset name available and not expose this meaningless container name. At least, we might think of a way for the header.xml to define global attributes and this dataset name could be put there.

All of the variables have the following attributes that are not in the original file and are for internal use:

• direction
• ferret_datatype
• definition
• dataset
• dataset_index

Coordinate variables have:

• dataset
• start
• end
• direction
• length

Migrated-From: http://dunkel.pmel.noaa.gov/trac/las/ticket/1514

[karlmsmith]

Attachment from @AnsleyManke on 10 Apr 2013 22:16 UTC

sampled.txt