Observation

[mbeaufils]

Source definition: The "Observation" concept from ISO19156

An observation is an act associated with a discrete time instant or period through which a number, term or other symbol is assigned to a characteristic. This act involves application of a specified procedure, such as a sensor, instrument, algorithm or process chain. The procedure may be applied in-situ, remotely, or ex-situ with respect to the sampling location. The result of an observation is an estimate of the value of a property of some feature; an observation is a property-value-provider for a feature-of-interest. Use of a common model allows observation data using different procedures to be combined unambiguously.

In conventional measurement theory (e.g.[12], [15], [16], [17], [19], [24]) the term “measurement” is used. However, a distinction between measurement and category-observation has been adopted in more recent work[13], [18], [25] so the term “observation” is used here for the general concept. “Measurement” may be reserved for cases where the result is a numerical quantity.

The observation itself is also a feature, since it has properties and identity. Observation details are important for data discovery and for data quality estimation. The observation could be considered to carry “property-level” instance metadata, which complements the dataset-level and feature-level metadata commonly provided via catalogue services (e.g. ISO 19115 or other community agreed one).

[mbeaufils]

This concept is proposed as the generic concept to which in-situ tests, laboratory analysis, etc... will derive from.

[neilchadwick-dg]

I am not quite ready to comment fully on this yet as I would like to get a clearer understanding of ObservationSupport (see my comments on #12 ). However, I thought it would be useful at this point to show everyone what we are currently doing in AGSi (the developing AGS ground model format - in beta). We seem to have a lot in common, but also some differences to maybe think about.

In AGSi we made a recent change to create an Observations group of objects. The intent of this group is to carry selected factual (or possibly interpreted) data within ground models. It is not intended as a replacement for the full AGS (factual) data format, which we suggest should still be used to carry the FULL factual data. The full documentation can be found here.

The AGSi Observation group objects are as follows:

As you can see, it is lightweight! The object names are hopefully mostly self explanatory.

We have agsiObservationSet as a top level parent object. This will most often correspond to a ground investigation, providing a home for the investigation metadata. However, it could be used to define some other fieldwork campaign, e.g. geological survey. I do not see an equivalent proposed so far - consider perhaps?

agsiObservationExpHole objects are used for the boreholes, pits, soundings etc. More or less equivalent to the factual AGS LOCA group. This has the bare minimum of attributes, but there is a way that extra data can be provided if need be. agsiObservationColumn is used for the individual (geological) units found in each hole.

We had geological field observations in our mind when we created agsiObservationPoint and agsiObservationShape, but these are general purpose objects that can accommodate almost anything.

[dponti]

What is (or is there) the distinction between an observation and a measurement? DIGGS uses these terms (probably incorrectly) to distinguish between observations that produce non-parametric or "subjective" or interpreted results such as visual-manual soil classification, stratigraphic or facies interpretations, or discontinuity descriptions, etc. (Observations) from what might be considered to be non-subjective results (typically numerical values) that derive as a result of some type of testing procedure or recorded by a sensor (Measurements).

To illustrate with a specific example: soil classification derived from particle size and atterberg limits tests on a soil sample would be a Measurement, whereas the classification of an interval of soil observed in a borehole based on the results of that sample testing (or from visual manual examination of cuttings) would be an Observation.

It seems that there would be some value in distinguishing between these two kinds of observations within our conceptual model or is this really all one thing?

[j-weil]

@dponti I agree to distinguish between those two. For the "observation", I would consider the locality, shape, extend... more important, e.g. a mapped structure or occurence of a certain material, according to an existing classification, may it be a logged interval or an exposure of the ground. For the "measurement" or test result, we are exchanging specific values for certain parameters at one location, that can be used e.g. for statistical analysis, and the shape of their representation is not so important.

[mbeaufils]

What is (or is there) the distinction between an observation and a measurement?

According to ISO19156, yes there is.

Here is the extended definition (in addition to what I proposed).

In conventional measurement theory (e.g.[12], [15], [16], [17], [19], [24]) the term “measurement” is used. However, a distinction between measurement and category-observation has been adopted in more recent work[13], [18], [25] so the term “observation” is used here for the general concept. “Measurement” may be reserved for cases where the result is a numerical quantity.

I can provide the links to (e.g.[12], [15], [16], [17], [19], [24]) and [13], [18], [25] if required.

[mbeaufils]

Discussion from the March 17 meeting:

• Observation made by humans, result = textual,
• Measurements made by devices, result = numeric value These are tendencies yet not strict, then decision to keep O & M them together and have the distinction on either the ObservedProperty, the Procedure.

[LucasoilCloud]

Another thing I would like to raise for this topic is regarding the measurements. Can a measurement be the result of an interpretation? For exemple, for a consolidation test, we measure a total mass and a settlement. With that we interpretate those values to have a stress and a void ratio. In this exemple the measurement is {Time, mass, settlement}. But can we consider {time, stress, void ratio} as a measurement or do we have to create another type of object "InterpretativeMeasurement"?

[dponti]

Thanks, @LucasoilCloud for opening up an important can of worms - that I assume we'll get to in more detail later.

From our perspective, when developing DIGGS and being mindful of the O&M concept for measurement observations (where the results are largely numeric and "objective"), the answer is yes you can blend interpretive and "measured" results within an Observation, and I believe you can keep everything together within one object (Observation or measurement object) that has the following component objects within it:

1) Domain (either location or time) 2) Results - consists of the result parameters measured and their values 3) Measurement procedure (see below)

The first two components of the Observation have the same properties regardless of the Observation being made. For a lab or in-situ test the Domain object contains the location(s) of the observation on the feature of interest, The Results would contain the list of properties measured and their values, calculated, estimated, etc. (the technique used to determine the result can be specified), as the "final" results of the observation. What "final" means are the results that define the characteristics of the feature of interest that you ultimately care about. These results depend on the test procedure, but in the case of many result properties, those properties can be obtained in a number of different ways (shear strength is a good example of this where there are multiple tests that can produce a shear strength result).

The measurement procedure object is unique to each test type. It would contain both a test specification - eg. the Procedure defined in #34 , but also, in the case of an Observation, would also contain metadata properties with values that are unique to this Observation instance, including the intermediate results (sort of sub-observations) that are collected and used to obtain the final result. It is very common with geotechnical tests that a procedure produces a set of values and those values are then used to obtain the ultimate result of the observation, that is applied in design. In many cases these intermediate values are important to report as a way to evaluate the test results. In other cases, such as for a moisture content, the pre-and post heating sample weights and tare weights (used to compute the moisture content), tend to be less important to report because the process is specified and simple, and most folks don't care to see that info (although it certainly can be reported) - they just want to know the moisture content.

For the test procedure you mentioned, DIGGS has s single Test object that is used for all measurements that occur within a spatial domain, that contains the three component objects described above. For the results of a consolidation test, we have defined the following "result" properties - these are done with controlled terms in a dictionary and not hard coded into the schema, so they can be easily added to: 1) vertical coefficient of consolidation 2) compression index 3) constrained vertical modulus 4) overconsolidation ratio 5) preconsolidation pressure 6) recompression index

These values are recorded in the TestResult object of the Test as appropriate (part 2 above), along with the location domain (part 1 above). All of the other information recorded as part of the test are contained within a ConsolidationTest procedure object (part 3 above) within the master Test object. You can see the schema for this object here. Give a minute for the doc to load - you can click on the elements to get more detail. This object contains basic metadata such as the specific consolidation test type (eg. Method A, B, etc) and the interpretation procedure, and then the sequence of loading increments where within each one are reported the applied load, loading stress, deformation after the loading increment is completed, as well as a sequence of consolidation increments where the axial deformation vs. elapsed time is recorded for the increment. All of this we consider test "metadata" - intermediate results that are specific to the test procedure that are used to generate the Observation results. The consolidation test is complex in that there are a lot of intermediate direct observations (eg.measured deformation) and calculations/estimates made that lead to the final result. Other types of test procedures might be very simple.

So in the DiGGS context: Test is the Observation TestResult contains the results of the Observation, along with the domain (location) ConsolidationTest is the procedure performed (with all of its metadata) to obtain the results.

Handling the information this way lends a consistent structure to all measurements, there is one place to look for the Observation results and one place to look to find out how those results were obtained. It also allows for a way to transfer Observation results (for legacy data for example) where the procedure and its metadata are unknown or not available.

There may be a way of standardizing the Test procedure object so that it is more consistent from test to test in its structure, but we never figured out how to get there without generalizing the object so much that one loses the "standard".. So for now we're stuck with creating an object for each class of test procedure...

[neilchadwick-dg]

@LucasoilCloud. Fair enough to raise this but I think I generally agree with @dponti . Whether an observation is, strictly speaking, a measurement or an interpretation (often based on measurement) is not important. It is the test procedure definition and metadata that will provide the information that the recipient needs.

Example: Liquid Limit - one of our most fundamental tests. The LL reported is an interpretation based on a set of measurements made during the test. So is it interpretation or measurement? I don't care. Its a Liquid Limit (observation) result. That is all I need to know.

@dponti - the question of what data is considered a 'result' and what is just 'metadata' will be an area to explore. For example, in the consolidation tests we often want to plot the void ratio vs pressure stage data during interpretative process - so should this be part of the results? FYI AGS handles both the 'results' and metadata' the same way , so we don't need to worry about it :-).

This may belong somewhere else, but lets say it here for now...

Where we will need to differentiate is with what comes after the observation when we carry out interpretation, i.e. when we summarise and analyse the data with the intent of coming up with a single value (or a profile of values, e.g. a design line) that may be used in design, usually a characteristic value. This is outside the scope of AGS (factual), but we have confronted it in AGSi - we came up with the concepts of properties (data from observations) and parameters (the value/profile to be used in design), and have assigned different objects to these.

Note that when I say to be used in design, there could be different flavours of this. For example, the values presented in a EC7 GIR may be provisional, perhaps even a range, that must be refined/confirmed before they can be used in a design. Alternatively, later in the process (e.g. EC7 GDR) values that can be directly used for a specific design may be provided. Conceptually, I don't see difference between these. They differ only in respect of the object specific use case, which I would envisage being provided as attribute data.

[mbeaufils]

Extract from the OMS specification

[Didymograptus]

An observation is an act associated with a discrete time instant or period through which a number, term or other symbol is assigned to a characteristic.

Unless I'm missing something, I think that there is a problem with this definition. Why is it only associated with a discrete time instant or period?

This may be appropriate for some things, but can you imagine drill crews, or engineers noting observations in a borehole or trial pit recording everything on a time basis? How accurate or vague can that time range be? Why is it always important? Why isn't an observation primarily associated with a spatial location and a time if that is important?

To create a model I need spatial data attributed to the result of an observation.

I suggest that the definition of an Observation be changed to:

An observation is an act associated with a obtaining a result.

and Result be:

A value or values (numeric or alphanumeric) reported as the output from an observation.

Leave out the spatial element and time date in the definition, this will be implicit in the standard used for reporting.

[mbeaufils]

Please remember OMS is designed for a lot of use cases (meteorology, military, space), not only geoscience.

I agree that for our geomodeling purpose location is important and so we will most of the time associate some position to our observations. Yet I am also thinking of some use cases where the location of the observation is not so important. Eg: for lab test and analysis (not to be confused to the location where the sample has been collected which is of course relevant).

Also about date/time, in a changing world I imagine this parameter is necessary. Of course the required granularity or precision of the date/time expression can be adjusted per use case.

[Didymograptus]

Is the time important for lab tests wrt geomodelling?

I guess I'm struggling with the approach being taken. My way of working is to define things that meet the need, in the most generic and simplest level. Then see if there is an existing definition that will fit. We are looking at existing definitions and trying to make them fit, without defining what we actually need.