Extensions for airplanes and drones

[davidraleigh]

Haven't been a part of planning in a while, so maybe this is already under discussion, but it seems like the only fields that should be included in sat are orbit_state and relative_orbit. Everything else was better off staying in an electro-optical message. If you fly a plane or a drone, now you're forced to split your elctro-optical information between eo and sat, and most of your relevant information must be contained in a sat object.

It honestly seems like a mess to make of things for two fields.

[matthewhanson]

The division of sat information out of EO was because all those fields were also present in the SAR extension as they all specific to satellite geometry and movement. It made sense to have a separate extension that could be used by both EO and SAR, and for pointcloud, full-motion video or other future content extensions for that matter. Those extensions relate to the type of data, whereas the sat extension is about the collection of that data.

At the time we also talked about making a Drone extension, and I think there's still a need for it, but we just didn't have any good examples of what fields would be needed. I don't think they are the same fields as in the sat extension, even though there might be some similar fields. The geometries that are of interest for drones are not the same ones for satellites, and they also may be measured separately. In satellite geometries we measure azimuth angles from north and off nadir angles. For drones I think the main angles of interest are going to be specified differently, such as relative to aircraft flight direction.

At any rate, we still need some good examples of drone geometry metadata that would be useful in STAC, and maybe it makes sense to reorganize things differently. Maybe they do belong in a single extension.

[davidraleigh]

I see how the overlap with sar would result in folding those into another more generic field that wasn't just eo.

Maybe instead of sat it should be named sensor or aerial-sensor. As far as I can tell these sat fields are relevant to aerial surveys:

sat:off_nadir_angle number  The angle from the sensor between nadir (straight down) and the scene center. Measured in degrees (0-90).
sat:incidence_angle number  The incidence angle is the angle between the vertical (normal) to the intercepting surface and the line of sight back to the satellite at the scene center. Measured in degrees (0-90).
sat:azimuth_angle   number  Viewing azimuth angle. The angle measured from the sub-satellite point (point on the ground below the platform) between the scene center and true north. Measured clockwise from north in degrees (0-360).
sat:sun_azimuth_angle   number  Sun azimuth angle. From the scene center point on the ground, this is the angle between truth north and the sun. Measured clockwise in degrees (0-360).
sat:sun_elevation_angle number  Sun elevation angle. The angle from the tangent of the scene center point to the sun. Measured from the horizon in degrees (0-90).

[cholmes]

@davidraleigh - you definitely know drones / planes better than us, so thanks for sounding in.

I think our main question on those fields, as @matthewhanson alluded to, was whether drone / aerial defines them in the same way as satellites. We thought they define them more relative to the aircraft, instead of from north. If you can confirm that the definitions of those sat fields are the exact same for aerial / drone then I definitely think a sensor naming makes sense, and we could put the orbit ones in sat. But if they are defined differently then a different name makes sense.

I'm curious on your complaint about splitting between eo and sat, with most relevant info in sat. What is the problem with that? That case is true for satellite data now. But we've also put them into a common place, so the prefix just is gives context. But I'm curious if you see a downside.

It would be good to get up a real public catalog with drone / aerial data.

[davidraleigh]

Spoke with a contact from Ursa who told me that there was no need for sun information in SAR data analysis. So moving sat:sun_azimuth_angle and sat:sun_elevation_angle out of electro-optical extension for the benefit of SAR doesn't seem to make much sense. Maybe other SAR users would care to comment?

updated recommendation: eo: [gsd, cloud_cover, bands, sun_azimuth_angle, sun_elevation_angle] sat: [orbit_state, relative_orbit] sensor: [off_nadir_angle, incidence_angle, azimuth_angle]

Could swap sensor name with scene_angles or capture_angles or something like that. Maybe even aerial_sensor...

I'm speaking with more aerial imagery users and providers later this week.

[davidraleigh]

I spoke with a VP/engineering lead at Near Map. And the summary was that there wasn't any clearly defined consensus on how to describe off nadir / oblique angle imagery within the community. That as far as he knew, there was no such definition within ASPRS or other photogrammetry imaging research organizations.

Our definition of off_nadir coincided with the way that Near Map was defining off nadir / oblique angle imagery. As for azimuth, that coincided with the Near Map definitions for cardinal directions for imaging (they use NSEW to define the direction of an image, and I believe images taken outside of that are just labeled as non-cardinal direction imagery). The STAC definitions for off_nadir and azimuth would work well for their imagery definitions.

I aim to speak with a photogrammetry researcher in the Geodesy department at OSU, A researcher at NCALM, and a few aerial imagery users from different companies to round out the comments from Near Map.

[cholmes]

Thanks a ton for the research on this @davidraleigh - the near map reference is really great to have. The conclusion was that if near map were to adopt STAC they could easily translate what they're doing into our off_ndair and azimuth definitions? If so that's great (and we should try to get them to implement STAC :) )

I think your proposed eo and sat groupings make sense - and yes, I can't see why SAR would need sun angles. The sensor name I'd say is too broad - lots of other types of data are gathered with different sensors that won't have angles. Like you could see on the ground soil data gathered by a sensor. overhead_sensor would make sense to me - I've heard sat + aerial referred to as overhead imagery. Or capture_angles also seems quite descriptive to me.

I think the other option is to just upgrade them to common metadata with no prefix. The cross domains and have wide use.

@geospatial-jeff - do you have any insight on angles? As you all are working with aerial imagery I believe.

[davidraleigh]

@cholmes I totally agree that either overhead_sensor or capture_angles (maybe abbreviated as os or ca?) are better descriptors than something too vague. Quick question and I'll stop being a pain; is the _angle suffix necessary?

I know maybe we don't need more information about this, but I got an update from an image scientist from Descartes Labs and a photogrammetry research professor from the OSU geodesy department. Both confirmed that the overhead_sensor angles we are proposing are well understood and used interchangeably for drones and aerial imagery.

Near Map was excited about the standard and particularly about the point cloud work, so I'll stay in touch with them about what they might be able to adopt.

[geospatial-jeff]

From my experience working with aerial providers (Nearmap included!) those angles are consistent across aerial/drone for images projected in a geodetic coordinate system. The one exception I can think of is LTP coordinate systems which are typically used for oblique imagery but would be better off in their own extension (if that ever becomes needed) since LTP is a fundamentally different way of referencing imagery.

[matthewhanson]

Thanks for doing some additional reaching out onthis @davidraleigh, it's great to get additional feedback and consensus.

I like the overhead sensor extension, and makes sense to have it separate from the sat extension if we can use it for drones. However, for an extension that contains various collection geometry angles, it seems like the natural place to include the sun angles as well. While it's true that this might not be useful for SAR data:

• Geometry angles are conceptually related, it's convenient and consistent to have them located in the same place.
• These are optional fields so if they aren't relevant they can be left out
• There is nothing in STAC requiring EO and SAR data be in separate Items. I could envision a co-collection product that might be in a single Item, or possibly a fused product that contains both SAR and optical.

The _angle prefix is nice for explicitly indicating these are angles, but I don't have a strong preference on this either. I also tend to like shorter names.

[izevaka]

@davidraleigh Thanks for bringing this to our attention. As a way of introduction, I look after the APIs and Integrations at Nearmap. I can offer some background about how we do oblique images in general terms.

There are a lot of metadata attributes associated with frame camera off nadir images that go beyond the azimuth and inclination angles. The combination of those attributes allows translation of real-world coordinates into image space with centimetre precision.

We happen to work with our photos in both UTM and LTP coordinates. Yes, the satellite positions come in as lat/lon, but they get converted into cartesian space pretty early on because that's what the ray tracing math requires.

The per photo attributes are:

• Camera position - X/Y/Z
• LTP Offset (for LTP space)
• UTM Zone information (for UTM space)
• Radial distortion parameters (three Konrady coefficients)
• Sensor dimensions X/Y (number of pixels)
• Sensor pixel size (metres per pixel)
• Camera orientation (three angles - omega, phi, kappa - this are rotations around the camera axis, rather than the world)
• Focal length

This is a rather simple model that assumes that the focal plane and the sensor plane are parallel and that the focal axis is in the middle of the sensor. This is not always the case and there are additional attributes that describe this situations.

I'm not familiar with pushbroom sensors. I suspect the maths will be quite different.

[davidraleigh]

@m-mohr and @izevaka

So this looks like the beginning of a separate aerial mapping extension, right?

[matthewhanson]

Maybe, it depends on if it's useful having this information within the STAC metadata rather than getting it from an original metadata file.

The primary goal of STAC is to facilitate search and discover, so the first question is are any of these fields something a user would search on? If not, including the source metadata as an asset might be better.

On the other hand, some extensions have been proposed, such as the projection extension, that contain info that is more useful during the processing because it is more efficient to get the data once rather than having to access additional files.

So do you think that these fields for aerial vehicles add value for searching, or enhance the ability to use it during processing as opposed to retrieving a source metadata file)?

[izevaka]

@matthewhanson With regards to searching, the only things that you'd realistically want to search on is by projected extent (derived piece of data, which I failed to mention in my earlier post). I.E give me all photos that cover this particular point/area. Having had a look at the spec, the detailed metadata fields seem way too specific for the standard, so I think having all that as vendor specific assets makes more sense.

Re projections - that makes sense as a core capability as different organisations store data in different projections.

Another thing to keep in mind is the epoch of spatial data. This is a glaring hole in current CRS specifications. By epoch I mean the date at which the coordinates were captured (typically from GPS). In practical terms this would normally include the ellipsoid (again, most commonly WGS84), the ITRF realisation (eg ITRF2014), and the epoch - date as fractional year. Having this info allows a proper conversion from Earth centric to plate centric co-ordinate system.

[matthewhanson]

I believe this has been resolved by splitting off fields from eo and sar into the sat and view extensions.

There were other potential issues raised here, please open as new issues if still pertinent.