Vertical Profiles - Borehole Data

[KoalaGeo]

Copying an issue from archived repo - https://github.com/covjson/specification/issues/92

Hi,

I'm exploring the feasibility of using CoverageJSON to share geotechnical borehole data, specifically AGS data - https://www.ags.org.uk/data-format/

This is a text file with a collection of groups of data, for 1 to n boreholes - see https://github.com/BritishGeologicalSurvey/pyagsapi/blob/main/test/files/example_ags.ags at it's most basic. They's a group for the project, group for location data (there's often more than 1 borehole on a project), group for sampling/geotchnical test data, group for geology etc.

The geology data might have values lie a lithology code & a description:

0 m = SOIL, Grass over top brown slightly gravelly sandy clay. Sand is fine to medium, gravel is subrounded to rounded and fine to medium 0.3 m = MUD, Greyish brown slightly gravelly slightly sandy silty clay with medium cobble content. Gravel is sub-angular to sub-rounded fine to coarse 1.0 m = CLAY, Black gravelly sandy clay 5 m = SAND, Reddish brown fine to coarse sand

A geotechnical test/sample might be like the below with multiple attributes:

0.5 m = 25, ES 0.8 m = 26, ES 1.11 m = 89, DS 1.26 m = 159, SP 3.45 m = 160, SD 4.87 m = 250, SR 4.90 m = 350, TO 5 m = 500, SO

A real .ags file example with multiple groups - https://github.com/BritishGeologicalSurvey/pyagsapi/blob/main/test/files/real/19684.ags

I need multiple z coordinates / irregular grid for each parameter & understand it's 1 z per domain. So I need to use a coverage collection for 1 borehole, with a coverage for each parameter which has unique z values.

Is there a way or having a collection of collections so 1 json could serve data from multiple boreholes/vertical profiles?

Example code for 1 borehole:

{
  "type" : "CoverageCollection",
  "domainType" : "VerticalProfile",
  "dct:identifier": "20374782",
  "dct:hasVersion": "4.0",
  "dct:rightsHolder": "MR E. O'BRIEN",
  "dct:Location": "GLAN-Y-NANT, LLANIDLOES",
  "dct:title": "Mount Severn- Environment Agency",
  "ags:contractor:": "W.B. & A.D. MORGAN LTD.",
  "ags:engineer": "R.MILLS & J. SCOTT",
  "dct:license": "https://creativecommons.org/licenses/by/4.0/",
  "parameters" : {
    "PTIM": {
      "type" : "Parameter",
      "description" : {
        "en": "Boring/Drilling Progress by Time"
      },
      "unit" : {
        "symbol" : "meters"
      },
      "observedProperty" : {
        "label" : {
          "en": "Boring/Drilling Progress by Time"
        }
      }
    },
    "GEOL": {
        "type" : "Parameter",
        "observedProperty" : {
          "id": "https://data.bgs.ac.uk/doc/Lexicon.html",
          "label" : {
            "en": "Lexicon"
          },
          "categories": [{
            "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/H4852.html",
            "label": {
              "en": "1st Terrace Deposits (River Severn)"
            }
          }, {
            "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/S945.html",
            "label": {
              "en": "2nd Terrace Of River Cam"
            }
          }, {
            "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/TSYC.html",
            "label": {
              "en": "36 Yard Coal (West Yorkshire)"
            }
          }]
        },
        "categoryEncoding": {
          "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/H4852.html": 0,
          "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/S945.html": 1,
          "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/TSYC.html": 2
        }
      }
  },
  "referencing": [{
    "coordinates": ["x","y"],
    "system": {
      "type": "GeographicCRS",
      "id": "http://www.opengis.net/def/crs/OGC/1.3/CRS84"
    }
  }, {
    "coordinates": ["z"],
    "system": {
      "type": "VerticalCRS",
      "cs": {
        "csAxes": [{
          "name": {
            "en": "Depth"
          },
          "direction": "down",
          "unit": {
            "symbol": "m"
          }
        }]
      }
    }
  }, {
    "coordinates": ["t"],
    "system": {
      "type": "TemporalRS",
      "calendar": "Gregorian"
    }
  }],
  "coverages": [
    {
      "type" : "Coverage",
      "domain" : {
        "type": "Domain",
        "axes": {
          "x": { "values": [-10.1] },
          "y": { "values": [-40.2] },
          "z": { "values": [ 5, 8, 14 ] },
          "t": { "values": ["2013-01-13T11:12:20Z"] }
        }
      },
      "ranges" : {
        "PTIM" : {
          "type" : "NdArray",
          "dataType": "float",
          "shape": [3],
          "axisNames": ["z"],
          "values" : [ 43.7, 43.8, 43.9 ]
        }
      }
    }, {
      "type" : "Coverage",
      "domain" : {
        "type": "Domain",
        "axes": {
          "x": { "values": [-10.1] },
          "y": { "values": [-40.2] },
          "z": { "values": [ 4, 7, 9 ] },
          "t": { "values": ["2013-01-13T12:12:20Z"] }
        }
      },
      "ranges" : {
        "GEOL" : {
          "type" : "NdArray",
          "dataType": "float",
          "shape": [3],
          "axisNames": ["z"],
          "values" : [ 0, 1, 2 ]
        }
      }
    }]
  }

[jonblower]

At the moment I don't think we have a way of having collections of collections. So you'd either have to "flatten" the collection somehow (maybe using some kind of identifier to indicate which borehole each coverage came from) or you'd have to use multiple json files.

To expand on the first option, I mean something like:

{
   "type": "CoverageCollection",
   ...
   "coverages":
   [
      {
         "type": "Coverage",
         "ex:borehole_id": "abcd123",
         ... 
         "ranges": {
            "PTIM": { ...}
         }
      },
      {
         "type": "Coverage",
         "ex:borehole_id": "abcd123",
         ... 
         "ranges": {
            "GEOL": { ...}
         }
      },
      {
         "type": "Coverage",
         "ex:borehole_id": "pqrs456",
         ... 
         "ranges": {
            "PTIM": { ...}
         }
      },
   ]
}

Maybe not an ideal solution but it fits the spec.

[KoalaGeo]

@jonblower thanks for that suggestion, good idea.

One further complication for boreholes which aren't vertical can you see how they could be accomodated? Additional coverages for azimuth and inclination of the hole?

[jonblower]

@KoalaGeo great, glad it was useful. Non-vertical boreholes are an interesting use case! Yes, you could have additional coverages to hold the data you need to describe the borehole itself. Maybe these could be cast on to a consistent set of domain coordinates so you could record them all as parameters of the same coverage?

You could use z as the domain coordinate, and make other things a function of z. Or you could potentially use another coordinate (e.g. the distance through the borehole) and make z (and everything else) a function of that. If you think of data as being primarily a function of depth (with azimuth and inclination as secondary concerns) then maybe the first approach is best.

[chris-little]

@KoalaGeo Non-Vertical Boreholes are very similar to weather sonde ascents, and marine bathythermographs descents, etc. I consider them, in essence, 3D or 4D trajectories. Either (x,y,z) or (x,y,z,t) for every data point along the path.

[KoalaGeo]

Good analogy @chris-little

In Oil & Gas context it's simplier as you'd be more likely to have measures say every cm along it's length, as it's often 1 piece of equipment taking all measurements.

In geotechnical or mineral exploration context the z intervals are more variable, 1 tool - azimuth & inclination, geologist - rock types, physical properties, samples all which will have different z (and they can be From or might be From - To....).

My understanding: I'd be looking at a Coverage Collection, with both Trajectory and Vertical Profile domains (can I mix domains in a collection?). The VerticalProfile would have multiple coverages for each measure (geology/porosity/magnetic suseptability etc).

Or would I need 2 files, Trajectory.covjson & Vertical_Profile_collection.covjson

[KoalaGeo]

Currently Association of Geotechnical & Geoenvironmental Specialists have the ags data format which does this. O&G industry use .LAS files Or user share 3 CSVs, collars, inclination/azimuth & downhole (from-to) measures/decriptions

[jonblower]

@KoalaGeo you're free to mix domain types in a Collection (but then you'd have to declare the domain type of each individual coverage rather than factoring out to the top level).

As to how you model the domain (as a vertical profile or trajectory, or indeed some other domain type), I think it depends how important the "non-verticality" is. If the borehole is basically vertical, but you want to record some information on deviations at specific depths (which might not be important for some applications), you might want to use a vertical profile. But if you really want to know the exact path of the borehole through 3D space (i.e. the x, y and z coordinate of each measurement point) then a Trajectory or MultiPoint may work better.

By analogy with sondes and XBTs - these are not strictly vertical profiles as they will not ascend in an exact vertical path. But for many applications the deviation from vertical doesn't matter and the approximation as a profile is OK.

[KoalaGeo]

This is all good, I wasn't sure if I was trying to crowbar data into a standard which really wasn't appropriate!

If using trajectories that then makes it more OGCAPI-EDR appropriate rather than OGCAPI-Coverages @chris-little ?

[jonblower]

Just to note that the Trajectory type allows for time variation from point to point in the domain (imagine a ship sailing a course and making measurements as it goes along). If you don't care about the time variation you could use MultiPoint if you want to record the x, y and z of each point.

Regarding your question, you might use the EDR API to request the coverage data, but use CoverageJSON as the return format. So I think both are relevant.

[KoalaGeo]

Hi Slightly amended covjson to keep it in one thread

{
    "type" : "CoverageCollection",
    "dct:identifier": "20374782",
    "dct:hasVersion": "4.0",
    "dct:rightsHolder": "MR E. O'BRIEN",
    "dct:Location": "GLAN-Y-NANT, LLANIDLOES",
    "dct:title": "Mount Severn- Environment Agency",
    "ags:contractor:": "W.B. & A.D. MORGAN LTD.",
    "ags:engineer": "R.MILLS & J. SCOTT",
    "dct:license": "https://creativecommons.org/licenses/by/4.0/",
    "parameters" : {
      "PTIM": {
        "type" : "Parameter",
        "description" : {
          "en": "Boring/Drilling Progress by Time"
        },
        "unit" : {
          "symbol" : "meters"
        },
        "observedProperty" : {
          "label" : {
            "en": "Boring/Drilling Progress by Time"
          }
        }
      },
      "GEOL": {
          "type" : "Parameter",
          "observedProperty" : {
            "id": "https://data.bgs.ac.uk/doc/Lexicon.html",
            "label" : {
              "en": "Lexicon"
            },
            "categories": [{
              "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/H4852.html",
              "label": {
                "en": "1st Terrace Deposits (River Severn)"
              }
            }, {
              "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/S945.html",
              "label": {
                "en": "2nd Terrace Of River Cam"
              }
            }, {
              "id": "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/TSYC.html",
              "label": {
                "en": "36 Yard Coal (West Yorkshire)"
              }
            }]
          },
          "categoryEncoding": {
            "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/H4852.html": 0,
            "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/S945.html": 1,
            "https://data.bgs.ac.uk/id/Lexicon/NamedRockUnit/TSYC.html": 2
          }
        }
    },
    "referencing": [{
      "coordinates": ["x","y"],
      "system": {
        "type": "GeographicCRS",
        "id": "http://www.opengis.net/def/crs/OGC/1.3/CRS84"
      }
    }, {
      "coordinates": ["z"],
      "system": {
        "type": "VerticalCRS",
        "cs": {
          "csAxes": [{
            "name": {
              "en": "Depth"
            },
            "direction": "down",
            "unit": {
              "symbol": "m"
            }
          }]
        }
      }
    }, {
      "coordinates": ["t"],
      "system": {
        "type": "TemporalRS",
        "calendar": "Gregorian"
      }
    }],
    "coverages": [
      {
        "type" : "Coverage",
        "domainType" : "VerticalProfile",
        "ex:borehole_id": "abcd123",
        "domain" : {
          "type": "Domain",
          "axes": {
            "x": { "values": [-10.1] },
            "y": { "values": [-40.2] },
            "z": { "values": [ 5, 8, 14 ] },
            "t": { "values": ["2013-01-13T11:12:20Z"] }
          }
        },
        "ranges" : {
          "PTIM" : {
            "type" : "NdArray",
            "dataType": "float",
            "shape": [3],
            "axisNames": ["z"],
            "values" : [ 43.7, 43.8, 43.9 ]
          }
        }
      },
      {
        "type" : "Coverage",
        "domainType" : "VerticalProfile",
        "ex:borehole_id": "abcd123",
        "domain" : {
          "type": "Domain",
          "axes": {
            "x": { "values": [-10.1] },
            "y": { "values": [-40.2] },
            "z": { "values": [ 4, 7, 9 ] },
            "t": { "values": ["2013-01-13T12:12:20Z"] }
          }
        },
        "ranges" : {
          "GEOL" : {
            "type" : "NdArray",
            "dataType": "float",
            "shape": [3],
            "axisNames": ["z"],
            "values" : [ 0, 1, 2 ]
          }
        }
      },
      {
        "type" : "Coverage",
        "domainType" : "VerticalProfile",
        "ex:borehole_id": "abcd456",
        "domain" : {
          "type": "Domain",
          "axes": {
            "x": { "values": [-10.1] },
            "y": { "values": [-40.2] },
            "z": { "values": [ 5, 8, 14 ] },
            "t": { "values": ["2013-01-13T11:12:20Z"] }
          }
        },
        "ranges" : {
          "PTIM" : {
            "type" : "NdArray",
            "dataType": "float",
            "shape": [3],
            "axisNames": ["z"],
            "values" : [ 43.7, 43.8, 43.9 ]
          }
        }
      },
      {
        "type" : "Coverage",
        "domainType" : "VerticalProfile",
        "ex:borehole_id": "abcd456",
        "domain" : {
          "type": "Domain",
          "axes": {
            "x": { "values": [-10.1] },
            "y": { "values": [-40.2] },
            "z": { "values": [ 4, 7, 9 ] },
            "t": { "values": ["2013-01-13T12:12:20Z"] }
          }
        },
        "ranges" : {
          "GEOL" : {
            "type" : "NdArray",
            "dataType": "float",
            "shape": [3],
            "axisNames": ["z"],
            "values" : [ 0, 1, 2 ]
          }
        }
      }
    ]
    }

[jonblower]

@KoalaGeo - this looks good to me (although I guess you could factor out the domainType to the Collection level, as they are all vertical profiles).

By the way, have you tried validating your CovJSON against the schema (https://github.com/covjson/covjson-validator)?

[KoalaGeo]

Not tried the validator before, I'll have a go.

Is ithere a plan to get the validation also built into https://github.com/opengeospatial/teamengine in the future? Part of the rest of OGC compliance tests?

[jonblower]

Is ithere a plan to get the validation also built into https://github.com/opengeospatial/teamengine in the future?

@KoalaGeo that's an interesting question. @ghobona, is that something we should look into?

[chris-little]

@KoalaGeo We have discusssed making CoverageJSON a full OGC standard, as opposed to a Community Standard. A full standard would eventually have a CITE test.

Don't hold your breath though. See Issue #125

[jonblower]

@KoalaGeo can we close this ticket now or is there anything outstanding? I've made a note of your helpful suggestion about TeamEngine in #125.

[KoalaGeo]

Hi @jonblower no, think we can close it - was a very useful discussion. Will go away and see if we can make a demo borehole DB > OA-EDR service.