Create timeSeriesProfile template

[GoogleCodeExporter]

timeSeriesProfile template should accomodate
1. upward/downward/sideways looking ADCP moored on fixed structure (bottom/dock 
etc) or on a moored buoy (e.g. at the interface between the buoy hull and the 
water looking downward or midway down the mooring line looking up/down/sideways)

2. Moored buoy with a string of e.g CT or Oxygen sensors but where the sensors 
are not homogeneously spaced.  e.g. at depth 1 there is a CT sensor, at depth 2 
there are CT and DO sensors, at depth 3 there is just a T sensor etc etc. For 
context, an OceanSITES deep water buoy might have 100 different sensors spread 
out over 5000m of mooring line, though 20-30 sensors is probably more realistic.

3. All timeSeriesProfiles should support sensors above and below the water just 
like the timeSeries template.

Original issue reported on code.google.com by dpsnowde...@gmail.com on 19 Feb 2013 at 7:41

[GoogleCodeExporter]

Two draft examples of time series profile have been created in collaboration 
with Shane StClair:

https://code.google.com/p/ioostech/source/browse/trunk/templates/Milestone1.0/SW
E-ADCP-TimeSeriesProfile_Alt1.xml
https://code.google.com/p/ioostech/source/browse/trunk/templates/Milestone1.0/SW
E-ADCP-TimeSeriesProfile_Alt2.xml

Both use the SWE data array to express the bin data and bin depths. 
Alt1 uses the NetCDF concept of shared dimension to loosely couple the bin 
depths to the bin data.
Alt2 makes the relationship more explicit via the use of a data choice element 
inside the data array.

Please consider both alternatives.

We can quickly extend this template to apply it to a string of sensors.

Original comment by stu3b3 on 1 Mar 2013 at 4:00

[GoogleCodeExporter]

After some thought and reflection Kyle, Shane and I support Alt 2 which Shane 
developed. Hopefully this option will be in alignment with the SWE requirements 
discussed in Issue#52.

Need to wrap this up this week for Milestone 1.0

Please make comments by close of business Thursday. 
We will be finalizing the draft templates for Milestone 1.0 on Friday unless 
someone asks to stop the presses.

David

Original comment by stu3b3 on 4 Mar 2013 at 6:33

• Changed state: Started

[GoogleCodeExporter]

Just for reference, this template should also support homogeneously-spaced 
profiles measuring the same variables everywhere. I suppose that's the simplest 
case in Derrick's use case #2 (in Comment 1). This situation comes up in data 
derived from CTD casts and mechanically lowered/raised profiling sensors.

Alt 2 seems horribly verbose, specially for profiles with many bins (say, >10). 
If the SWE 2 issue discussed in Issue#52 doesn't conflict with Alt 1 (as I 
think Alex has implied), I'd vote for Alt 1.

Question, I think for both alternatives: do they assume that a single time 
stamp applies to all bins? I think that's what I'm reading. In most cases where 
the same set of sensors are doing the measurements in situ (my case #2 
sub-case), like CTD's casts, that's generally assumed, but there is an actual 
time separation that may span a good chunk of time (> 30min ?) for deep 
profiles, and it'd be good to be able to accommodate the actual bin-specific 
times.

BTW, I'm not convinced that this Issue (developing this template) should be in 
Milestone 1.0 (rather than 1.1), but that's a different point.

Original comment by emilioma...@gmail.com on 5 Mar 2013 at 1:41

[GoogleCodeExporter]

I will give some thought as to how we could make Alt 1 compliant without 
bloating the result block. Your point about lowered profiles is well taken. In 
the context of a coastal ADCP with 50 bins Alt 2 is fine. For a deep water 
moored profiler with 4000 bins it is not tenable. 

Re sample time: It is straight forward to add a field with a reference time in 
seconds as in page 78 of OGC 08-094r1. Creating a consistent and correct 
template for CTD data is quiet a bit of work as it would raise many other 
issues. Can we complete Milestone 1.0 with just the ADCP template assuming we 
resolve Alt 1/2 issue?

What data do we need to start using when Milestone 1.0 wraps up?

David

Original comment by stu3b3 on 5 Mar 2013 at 3:13

[GoogleCodeExporter]

Both of these samples have the bin depth info hard coded in the SWE in values 
elements.  This does not play well in real life.  I have seen bottom mounted, 
upward looking ADCPs as well as downward looking ADCPs mounted at a fixed spot 
on an underwater structure.  In both of these cases the bin depth varies with 
the tides.  

Original comment by mike.gar...@gmail.com on 5 Mar 2013 at 3:22

[GoogleCodeExporter]

Hi Mike

The intent was that by expressing the bin location as height relative to the 
sensor that we could deal with what ever issues might arise by allowing the 
sensor location to be a arbitrarily complex observed value in each record. Is 
that not the case? It has been a while since I worked with ADCP data directly - 
does the range from the sensor actually change for each bin over time? If so, I 
see no alternative but to encode the bin location with the speed and direction 
which is perfectly acceptable using these templates. 

David

Original comment by stu3b3 on 5 Mar 2013 at 3:37

[GoogleCodeExporter]

Emilio, for Alt2 I'm assuming that the horrible verbosity you're referring to 
is in the repeated bin field descriptions and not in the values block. The 
values block could be tightened up a bit by shrinking the bin DataChoice item 
names to just b1, b2, etc. It's true that huge numbers of bins will add a lot 
of bloat to the bin descriptions. Maybe for Alt2 we should include the option 
of omitting the bin DataChoice and allowing the repeated encoding of the bin 
dimensions in the values block to allow some flexibility for both extremes.

Regarding your single time for all bins question, with Alt2 we could make the 
DataArray a variable length array so that only bins with measurements at a 
certain time would be encoded. This could work with both the DataChoice bins 
and values encoded bins option. The values block would look something like:

2009-05-23T00:00:00Z,0.002,0.005,0.017,18.02,11928,2,bin1,360.0,25.0,bin3,358.0,
23.0
2009-05-23T00:00:00Z,0.002,0.005,0.017,18.02,11928,1,bin2,359.0,24.0
2009-05-23T00:00:00Z,0.002,0.005,0.017,18.02,11928,1,bin3,358.0,23.0
2009-05-23T01:00:00Z,0.002,0.005,0.017,18.02,11928,2,bin1,360.0,25.0,bin2,000.0,
24.0
2009-05-23T02:00:00Z,0.002,0.005,0.017,18.02,11928,3,bin1,360.0,25.0,bin2,000.0,
24.0,bin3,358.0,23.0
2009-05-23T03:00:00Z,0.002,0.005,0.017,18.02,11928,3,bin1,360.0,25.0,bin2,359.9,
24.0,bin3,358.0,23.0
2009-05-23T04:00:00Z,0.002,0.005,0.017,18.02,11928,2,bin2,359.9,24.0,bin3,358.0,
23.0
2009-05-23T00:00:00Z,0.002,0.005,0.017,18.02,11928,1,bin1,360.0,25.0
2009-05-23T05:00:00Z,0.002,0.005,0.017,18.02,11928,3,bin1,360.0,25.0,bin2,359.9,
24.0,bin3,358.0,23.0

Original comment by srstcl...@gmail.com on 5 Mar 2013 at 3:59

[GoogleCodeExporter]

I completely missed the sensor relative portion.  I am so used to reporting 
currents observations with bin info converted to a depth below sea level. 
Though we receive all the needed bin configuration info, we compute and store 
the depth of each bin in our SOS database.  That means our current SOS database 
has no access to the info needed to populate the bin info section of the 
template.   On the other hand, it is simpler for the user to see a report at a 
given depth without having to do more calculations.

Typically, the bin configuration does not change with a single instrument 
deployment.

Original comment by mike.gar...@gmail.com on 5 Mar 2013 at 4:03

[GoogleCodeExporter]

Regarding Shane's Comment 7:
- Yes, the verbosity I was referring to is in the repeated bin field 
descriptions and not in the values block. Sorry for not clarifying that.
- Issue of single time for all bins: in general, in the cases I'm familiar with 
where a single time is not assigned to the whole profile, each bin would have a 
different time, which may vary across adjacent bins by <1 min.

Re: Comment 4, making Alt 1 compliant without bloating the result block: I had 
confused what Shane reported on Issue 52, Comment 1. I see now that the bin 
description block in Alt 1 would not be compliant.

Regarding this more general comments:
Can we complete Milestone 1.0 with just the ADCP template assuming we resolve 
Alt 1/2 issue? What data do we need to start using when Milestone 1.0 wraps up?

That's a question for Derrick. But these sort of complexities are why I suggest 
deferring the timeSeriesProfile template altogether to Milestone 1.1, and 
instead spend time cleaning up 1.0 and getting it out the window.

Original comment by emilioma...@gmail.com on 5 Mar 2013 at 4:30

[GoogleCodeExporter]

The division of the static and dynamic data in the SWE document made the 
implementation of the timeSeriesProfile relatively trivial. It appears that it 
will also be easily extensible to create the trajectory feature type.

https://code.google.com/p/ioostech/source/browse/trunk/templates/Milestone1.0/SW
E-SingleStation-TimeSeriesProfile.xml

This template should be general enough to accomodate all kinds of profile data 
including thermistor chains and ADCP's including accommodation for ragged array 
lengths. The model depends on the correct application of the profile-bin or 
profile-height arrays which contain the profile structure. The dynamic 
observation data is also stored in an array which contains an index reference 
to the profile-bin. 

This should take care of all the concerns about the previous template versions 
which were either too verbose in the encoded data block or in the SWE structure 
or both.

David

Original comment by stu3b3 on 1 May 2013 at 8:21

• Changed state: Fixed