Time ordering of Coverages in a Collection

[pzaborowski]

In the particle tracking use case and drift models, the usual outcome is set of:

• trajectories - from drift models they are identifiable along the time, something like NCEI Multiple mooring lines with stationary instruments at different depths across the mooring lines and the instruments measuring at different points in time. but without 'z' in dim.
• series of snapshots, each snapshot is a MultiPoint, each point has properties - good for raw observations, particles are not identifiable at source, also preference for visualization as it allows for present frame by frame.

Here is the covjson schema compliant second representation as CoverageCollection of MultiPoint, each Multipoint at different time: https://github.com/ILIAD-ocean-twin/data_access_api/issues/12

Traversing through all the Coverages/frames to know what are the timestamps and to order them does not feel like elegant. So example has additional time domain (not breaking the schema nor mentioned in spec as MAY for CoverageCollection) on the collection level where all the timestamps are listed and sorted. It is redundant to timestamps in the Coverages while allowing to construct timescale, potentiallyy faster (all timestamps in one block, no need to traverse all the Coverages, esp they does not have to be in order).

Is that a good case to say what is the domain of the coverage collection if exists? Is it a good candidate for #161 type profile?

[jonblower]

Thanks for your question and the supporting info! I think I understand the use case, and I can see that it is an attractive solution to "factor out" the time dimension into the CoverageCollection.

However, this does bring the potential for the information in the Collection to be inconsistent with its constituent Coverages, as the information in the Collection would be redundant. Although it is perhaps not elegant to traverse the Coverages, do you have reason to believe that this causes significant problems with efficiency? I would think that if the Coverages are all in memory, it would be just a few lines of code and a quick operation to extract the timestamps. You might even be able to do it with JSONPath, although I've never tried it!

[pzaborowski]

thanks, I may do some tests, BUT is it required that Coverages in the collection are in order? unlike timestamps in the dimension I have not found such requirement, which now I think maybe the most elegant option. Obviously it works for time, not any other dimension.

justification: my feeling is it is more resource-consuming to put them in order than traverse. You can make it also in one line of code, but it would either mean you have to build your own index of timestamps (redundant), sort them, and then refer to the objects by these timestamps - double the same computation OR move the whole objects around with one command, potentially waste of cpu as each is a bulk in memory.

[jonblower]

BUT is it required that Coverages in the collection are in order?

No, there is no ordering requirement on the Coverages in a Collection. But I could imagine having some kind of a tag on the Collection that asserts that the Coverages are in order of time. (This could be a kind of restrictive profile, as you suggest in your original post.)

I'm still not sure it would be worth the effort, unless there was evidence that the indexing/resorting process really is a bottleneck in a system. Would the Coverages not usually be in time order in your case anyway, as that would seem to be the natural ordering that a system would use?

[pzaborowski]

Yes, that would be natural ordering, while that assumption on the client application could be faulty:). ok, do not want to be stubborn on that. we'll put it in our case in the template description for the moment:) cheers

[jonblower]

Great! I think your proposal is a good idea, but I'm just a bit wary of introducing redundant information into the structure. If you face practical issues with the current spec, we can certainly return to this.

[chris-little]

@pzaborowski @jonblower If the coverages are expected to be ordered in time, surely then time is being used as a dimension/coordinate? Of course it could be a second or multiple time dimension, different from a declared time dimension. Or are the timestamps actually coverage values, along with other values, in a multivalue rangeset?

[jonblower]

@chris-little There is no dimension (of any kind) at the level of the collection, which is really just an array of coverages with a bit of extra metadata. The dimensions are on the Coverages, and of course at the level of the Coverage the time dimension is (probably) ordered.

[jonblower]

@pzaborowski there hasn't been further discussion of this issue for a while. Are you happy if we close this, and we can always re-open it if necessary? For summary, I think the options identified were:

1. Add a time dimension to the CoverageCollection structure, which lists the times of each individual Coverage. Downside is the possibility of inconsistency between the Collection and the Coverages.
2. Add a tag (e.g. "timeOrdered=true") to the CoverageCollection, which asserts to the client that the Coverages are in increasing order of time. Probably fairly simple, but still needs to be specified.
3. Do nothing, and use client-side logic to order (or verify ordering of) the Coverages.

Personally I'm in favour of doing nothing because I'm keen to keep the spec minimal, but I'm aware this isn't too helpful for you! If you would like to pursue one of the other options, it would be great if you could make a proposal.

[jonblower]

(For ease of future discovery, it might be worth renaming this issue to "Time ordering of Coverages in a Collection" or something like that.)