search

cf-convention / vocabularies

Issues and source files for CF controlled vocabularies
3 stars 1 forks source link

Standard names: Point clouds, DEMs and DTMs #239

Open lhmarsden opened 2 days ago

lhmarsden commented 2 days ago

I am working on a data flow for storing data in point clouds in CF-NetCDF files. A point cloud is a collection of data points in a three-dimensional space, typically representing the surface of an object or environment, where each point contains spatial coordinates and may include additional attributes such as colour, normals, or intensity. The data I am working on are mostly focused on natural features (valleys, glaciers etc). However, a point clouds can be used to map the surface of man-made objects too. A point cloud could equally include both - a valley may include buildings for example.

I think CF-NetCDF is a suitable format for this type of data; existing commonly-used formats are not FAIR compliant, as far as I am aware. DEMs and DTMs could be stored in the same way, with the featureType being point.

Here is how I envisage the file might look, with a few data variables added as an example:

Example

dimensions:
        point = 9968400 ;
        band = 186 ;
variables:
        float point(point) ;
                point:units = "1" ;
                point:long_name = "Arbitrary counter for number of points in the point cloud" ;
                point:standard_name = "number_of_observations" ;
                point:coverage_content_type = "coordinate" ;
        float band(band) ;
                band:units = "nanometers" ;
                band:long_name = "Spectral band" ;
                band:standard_name = "radiation_wavelength" ;
                band:coverage_content_type = "coordinate" ;
        int crs ;
                crs:crs_wkt = "PROJCRS[\"unknown\",BASEGEOGCRS[\"unknown\",DATUM[\"World Geodetic System 1984\",ELLIPSOID[\"WGS 84\",6378137,298.257223563,LENGTHUNIT[\"metre\",1]],ID[\"EPSG\",6326]],PRIMEM[\"Greenwich\",0,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8901]]],CONVERSION[\"UTM zone 33N\",METHOD[\"Transverse Mercator\",ID[\"EPSG\",9807]],PARAMETER[\"Latitude of natural origin\",0,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8801]],PARAMETER[\"Longitude of natural origin\",15,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8802]],PARAMETER[\"Scale factor at natural origin\",0.9996,SCALEUNIT[\"unity\",1],ID[\"EPSG\",8805]],PARAMETER[\"False easting\",500000,LENGTHUNIT[\"metre\",1],ID[\"EPSG\",8806]],PARAMETER[\"False northing\",0,LENGTHUNIT[\"metre\",1],ID[\"EPSG\",8807]],ID[\"EPSG\",16033]],CS[Cartesian,2],AXIS[\"(E)\",east,ORDER[1],LENGTHUNIT[\"metre\",1,ID[\"EPSG\",9001]]],AXIS[\"(N)\",north,ORDER[2],LENGTHUNIT[\"metre\",1,ID[\"EPSG\",9001]]]]" ;
                crs:semi_major_axis = 6378137. ;
                crs:semi_minor_axis = 6356752.31424518 ;
                crs:inverse_flattening = 298.257223563 ;
                crs:reference_ellipsoid_name = "WGS 84" ;
                crs:longitude_of_prime_meridian = 0. ;
                crs:prime_meridian_name = "Greenwich" ;
                crs:geographic_crs_name = "unknown" ;
                crs:horizontal_datum_name = "World Geodetic System 1984" ;
                crs:projected_crs_name = "unknown" ;
                crs:grid_mapping_name = "transverse_mercator" ;
                crs:latitude_of_projection_origin = 0. ;
                crs:longitude_of_central_meridian = 15. ;
                crs:false_easting = 500000. ;
                crs:false_northing = 0. ;
                crs:scale_factor_at_central_meridian = 0.9996 ;
        float longitude(point) ;
                longitude:units = "degrees_east" ;
                longitude:long_name = "longitude" ;
                longitude:standard_name = "longitude" ;
                longitude:axis = "X" ;
                longitude:valid_range = -180., 180. ;
                longitude:coverage_content_type = "coordinate" ;
        float latitude(point) ;
                latitude:units = "degrees_north" ;
                latitude:long_name = "latitude" ;
                latitude:standard_name = "latitude" ;
                latitude:axis = "Y" ;
                latitude:valid_range = -90., 90. ;
                latitude:coverage_content_type = "coordinate" ;
        float X(point) ;
                X:units = "meters" ;
                X:long_name = "X coordinate" ;
                X:standard_name = "projection_x_coordinate" ;
                X:grid_mapping = "crs" ;
                X:coverage_content_type = "coordinate" ;
        float Y(point) ;
                Y:units = "meters" ;
                Y:long_name = "Y coordinate" ;
                Y:standard_name = "projection_y_coordinate" ;
                Y:grid_mapping = "crs" ;
                Y:coverage_content_type = "coordinate" ;
        float Z(point) ;
                Z:units = "meters" ;
                Z:long_name = "Z coordinate" ;
                Z:standard_name = "height" ;
                Z:grid_mapping = "crs" ;
                Z:coverage_content_type = "coordinate" ;
        float red(point) ;
                red:units = "1" ;
                red:long_name = "red channel" ;
                red:coverage_content_type = "physicalMeasurement" ;
                red:coordinates = "latitude longitude" ;
        float green(point) ;
                green:units = "1" ;
                green:long_name = "green channel" ;
                green:coverage_content_type = "physicalMeasurement" ;
                green:coordinates = "latitude longitude" ;
        float blue(point) ;
                blue:units = "1" ;
                blue:long_name = "blue channel" ;
                blue:coverage_content_type = "physicalMeasurement" ;
                blue:coordinates = "latitude longitude" ;
        float intensity(point, band) ;
                intensity:units = "W m-2 sr-1 nm-1" ;
                intensity:long_name = "Spectral intensity" ;
                intensity:standard_name = "toa_outgoing_radiance_per_unit_wavelength" ;
                intensity:coverage_content_type = "physicalMeasurement" ;
                intensity:coordinates = "latitude longitude" ;

However, I am missing a few standard names.

Points and directions

point_elevation_above_geoid

Definition: A point is a single spatial measurement on an object's surface as measured in a point cloud or digital terrain model. elevation_above_geoid is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

Note: A point could be the ground surface or the surface of a manmade feature. Therefore, I don't think this is covered by existing standard names. One could imagine two points of different elevation at the same lat/lon - e.g. a bridge or an overhanging cliff.

Canonical units: m

sensor_to_point_direction_x_component

Definition: X-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "x" indicates a vector component along the grid x-axis, positive with increasing x.

Canonical units: 1 (dimensionless)

I envisage that this and the below 2 terms could be useful in other applications. Anywhere that it is interesting to know the direction between the sensor and a point of interest.

sensor_to_point_direction_y_component

Definition: Y-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "y" indicates a vector component along the grid y-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

sensor_to_point_direction_z_component

Definition: Z-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "z" indicates a vector component along the grid z-axis, positive with increasing z.

Canonical units: 1 (dimensionless)

surface_normal_vector_x_component

Definition: The X-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "x" indicates a vector component along the grid x-axis, positive with increasing x.

Canonical units: 1 (dimensionless)

surface_normal_vector_y_component

Definition: The Y-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "y" indicates a vector component along the grid y-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

surface_normal_vector_z_component

Definition: The Z-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "z" indicates a vector component along the grid z-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

Red, green and blue

Normalised RGB values can be on different scales, with different valid ranges:

I am unsure how to deal with this. Perhaps the user should be required to include valid_min and valid_max variables attributes? Or should CF specify which range the user should use? It is not obvious to me which range is best.

The wavelength that red, green and blue refer to can vary between different applications, and depends on the context. For the sake of interoperability, the wavelength referred to should be stated. I am not sure what is the best approach here. Perhaps a variable attribute? This is usually only used for quick preview purposes as far as I am aware, so maybe this isn't an issue. In my example I give the intensity of each band as a 2D variable, so one could compute their own RGB values if they wanted to. However, I think it is useful to include both.

normalised_red_wavelength_intensity_of_point

Definition: Intensity of electromagnetic radiation in the red wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the red component of the RGB colour model.

Canonical units: 1

normalised could be scaled

normalised_green_wavelength_intensity_of_point

Definition: Intensity of electromagnetic radiation in the green wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the green component of the RGB colour model.

Canonical units: 1

normalised could be scaled

normalised_blue_wavelength_intensity_of_point

Definition: Intensity of electromagnetic radiation in the blue wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the blue component of the RGB colour model.

Canonical units: 1

normalised could be scaled

github-actions[bot] commented 2 days ago

Thank you for your proposal. These terms will be added to the cfeditor (http://cfeditor.ceda.ac.uk/proposals/1) shortly. Your proposal will then be reviewed and commented on by the community and Standard Names moderator.

JonathanGregory commented 2 days ago

Dear Luke @lhmarsden

It is certainly CF-compliant to store a field of data at scattered points, as you suggest. Your point dimension and coordinate variable are a "discrete axis" (section 4.5). In addition, you can designate this arrangement with the attribute featureType="point" (sect 9.1, examples in appendix H1). Is this what you mean?

I don't think quantities at points need special standard names. They are the same quantities as in continuous geophysical fields, but sampled discretely. However, most of your quantities probably do need standard names. The first one can use the existing standard name of altitude, which means height above geoid.

Best wishes

Jonathan

lhmarsden commented 2 days ago

Thanks for your reply, Jonathan.

Yes, I think the discrete axis section is relevant here, and I agree that most of the variables don't need special standard names.

I was not sure about whether altitude was appropriate in this case or not. I was envisioning a scenario where we just have lat, lon and altitude. For example, a DEM. If we use the standard name altitude, is it clear that this is the altitude at the surface, or just some altitude at which measurements are taken? Maybe I am being pedantic here...

Luke

From: JonathanGregory @.> Sent: Tuesday, November 19, 2024 2:28:02 pm To: cf-convention/vocabularies @.> Cc: Luke Marsden @.>; Mention @.> Subject: Re: [cf-convention/vocabularies] Standard names: Point clouds, DEMs and DTMs (Issue #239)

Dear Luke @lhmarsdenhttps://github.com/lhmarsden

It is certainly CF-compliant to store a field of data at scattered points, as you suggest. Your point dimension and coordinate variable are a "discrete axis" (section 4.5https://cfconventions.org/cf-conventions/cf-conventions.html#discrete-axis). In addition, you can designate this arrangement with the attribute featureType="point" (sect 9.1https://cfconventions.org/cf-conventions/cf-conventions.html#_features_and_feature_types, examples in appendix H1https://cfconventions.org/cf-conventions/cf-conventions.html#point-data). Is this what you mean?

I don't think quantities at points need special standard names. They are the same quantities as in continuous geophysical fields, but sampled discretely. However, most of your quantities probably do need standard names. The first one can use the existing standard name of altitude, which means height above geoid.

Best wishes

Jonathan

— Reply to this email directly, view it on GitHubhttps://github.com/cf-convention/vocabularies/issues/239#issuecomment-2485714078, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOMNBFKFYT3SFGY6J6UTTLD2BM4F3AVCNFSM6AAAAABSBQZJ4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBVG4YTIMBXHA. You are receiving this because you were mentioned.Message ID: @.***>

JonathanGregory commented 2 days ago

Sorry, yes. There is a standard name of surface_altitude too, which is presumably the right one?

lhmarsden commented 2 days ago

This is close. Maybe this is okay. Maybe I am being pedantic. The definition is

The surface called "surface" means the lower boundary of the atmosphere. Altitude is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

If the point cloud is of a bridge, and includes the underside of the bridge, can that be considered the lower boundary of the atmosphere?

What is the protocol around adjusting definitions for existing standard names? The name itself does seem suitable, and it feels unnecessary to add a new name for this.

'The surface called "surface" ' seems to be a bit clumsy wording to me. Perhaps a more suitable definition would be

"surface" refers to the boundary between the atmosphere and an object. Altitude is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

Luke

ChrisBarker-NOAA commented 2 days ago

is Z -- elvation, height, ??? -- a property of a point at a 2D location, or is the point a 3D point?

        float Z(point) ;
                Z:units = "meters" ;
                Z:long_name = "Z coordinate" ;
                Z:standard_name = "height" ;
                Z:grid_mapping = "crs" ;
                Z:coverage_content_type = "coordinate" ;

Calling it a coordinate makes me think it's well, the 3rd coordinate of the point.

If the latter, then I think the properties should have:

            green:coordinates = "Z, latitude longitude" ;

And that makes this different than a DEM, which, IIUC, defines a surface, such that each location in the 2D, X-Y (lat-lon) space has a single elevation value.

I think a point cloud can have more than. one point at the same horizontal location -- i.e. on on top of the other.

Does this have any bearing on the standard name? I"m not sure it does, but we should make sure the examples are clear.

lhmarsden commented 2 days ago

I think this is a 3rd coordinate of a point. You are correct, I should update the coordinates variable attribute to state this.

For a point cloud, this is just the location of a point in space, all three dimensions.