Document geospatial support

[danielmewes]

We need API docs for:

• r.geojson
• toGeojson
• r.point
• r.line
• r.polygon
• r.circle
• distance
• intersects
• includes
• getIntersecting
• getNearest
• fill
• polygon_sub

We also have to extend the documentation for indexCreate.

Some details about indexCreate could still change before 1.15, but the other commands are pretty much fixed.

[chipotle]

Are these in a quick draft yet, or in comments in the source code or such?

[danielmewes]

Ok, here is a (hopefully complete) description of geometry features in ReQL. This is meant to serve as an internal reference and as a basis for @chipotle to write some actual user documentation (API docs and maybe an additional article for our documents?).

GENERAL

• Coordinates All geometry in RethinkDB lives on the surface of a sphere in three-dimensional space. As a consequence, coordinates are two dimensional. Positions on the sphere's surface are addressed by a pair of latitude and longitude (in this order. That is different from GeoJSON, which uses longitude, latitude). The latitude is a real (floating point) number between -90 and 90. The longitude is between -180 and 180. The longitudes 180 and -180 are equivalent, while latitudes -90 and 90 are actually on opposite sides of the globe (the one is at the south pole, the other at the north pole). See http://en.wikipedia.org/wiki/Geographic_coordinate_system for details.

• Geometric primitives and the GEOMETRY type ReQL supports the following geometric objects:

  • Points - just a single coordinate pair
  • Lines - A sequence of coordinate pairs, at least two of them.
  • Polygons - Exactly one outer shell, and an arbitrary number of holes. The shell is a line which first and last coordinate pair are equal, and which doesn't intersect with itself. The holes are like shells, but each hole must be completely contained in the shell minus all other holes. The set of points that defines the polygon is the set of all points "inside" the shell, with points inside of any of the holes cut out.

  A new ReQL pseudo type "GEOMETRY" is introduced to represent any of these geometric objects.

• Lines between two points In contrast to Euclidean geometry, "lines" between two points on the surface of a sphere are not straight. In ReQL a line between two points is given by the shortest path over the sphere's surface. Such lines are commonly known as "Geodesics". See http://en.wikipedia.org/wiki/Geodesic for details. It follows that two connected points must not be antipodal (e.g. one at lat = -90, the other one at lat = 90), or else the line connecting them is not defined. (ReQL might or might not trigger an error if such a line is constructed, depending on context. If it doesn't trigger an error, it picks one of the infinitely many possible geodesics that connect the points, and the results of intersection / inclusion tests will be undefined). Lines and Polygons are defined in terms geodesics. Note that we assume a perfect sphere for this. The earth is not a perfect sphere, but comes close to it. Especially when the distance between two points is short compared to earth's radius, the spherical geodesic will be practically identical to the shortest path assuming a (more exact) elliptical model of earth's geometry.

• Distances Distances between two points in ReQL are calculated assuming an ellipsoid earth. This is a refinement over the earth as a sphere model. The distance between two points is defined as the length of the shortest path along the ellipsoid's surface that connects both points. See http://en.wikipedia.org/wiki/Geodesics_on_an_ellipsoid for details. Computing such distances is not trivial. RethinkDB uses Karney's algorithm introduced in http://link.springer.com/article/10.1007%2Fs00190-012-0578-z This is a state of the art algorithm that is relatively fast, handles corner cases well and is highly exact.

  All ReQL functions that operate with distances accept two optional arguments: geo_system (geoSystem in JS): The reference ellipsoid to use. Possible values are: "WGS84" and "unit_sphere". The default is "WGS84", which is a common standard for earth's geometry http://en.wikipedia.org/wiki/World_Geodetic_System "unit_sphere" assumes a perfect sphere of radius 1 meter. unit: The distance unit to use. Possible values are: "m" - meters, "km" - kilometers, "mi" - international miles, "nm" - nautical miles, "ft" - international feet. The default is "m". All distances that are passed into a ReQL term that accepts this argument will be interpreted in the corresponding unit, and results - if they contain distances - will be given in the unit.

GEOSPATIAL INDEXES

• How to create one? A geospatial index is created similarly to a regular index, however with an the optional argument "geo" set to true. E.g. r.table('t1').indexCreate('geometry', {geo: true}) This command creates a geospatial index over the field geometry on table t1. Similar to regular indexes, geospatial indexes can be created over arbitrary deterministic ReQL functions. It's also possible to combine the multi flag for creating a multi index with the geo flag.
• Which operations are applicable? The following ReQL terms can operate with geospatial indexes: getIntersecting, getNearest (and of course indexStatus, indexWait, indexDrop, indexList) Trying to use a geospatial index with other terms such as getAll, orderBy or between results in an error. At the same time, getIntersecting and getNearest can only be used with a geospatial index. Using those commands without an index at all or with a non-geospatial (regular) secondary index results in an error.
• What happens with non-geometric objects? If there are documents in the table that have a value for the indexed field which is not of type GEOMETRY, those documents will be ignored by a geospatial index.
• What happens if I insert a geo object into a non-geo index? Documents that have a value of type GEOMETRY for a field indexed by a regular secondary index will be included in the index, but their ordering is rather arbitrary. In fact such documents will be sorted within the index as if they were just JSON objects.

ReQL TERMS

• r.geojson(geojson) : OBJECT -> GEOMETRY The r.geoJson term converts a given GeoJSON object into a ReQL GEOMETRY item. GeoJSON is a widely used standard for representing geometrical data. See http://geojson.org/ for more details. Out of the GeoJSON geometry types, r.geojson supports the Point, LineString and Polygon types. MultiPoint, MultiLineString and MultiPolygon are not supported. (a work-around is storing multiple single Points/Lines/Polygons in an array and using a geospatial multi-index for indexing them) The following additional restrictions exist: Only longitude/latitude coordinate pairs are supported. Cartesian coordinates are not. Furthermore, GeoJSON objects that specify an altitude in addition to longitude and latitude are rejected by r.geoJson. GeoJSON objects that specify their own coordinate reference system (CRS) are rejected as well.

  Note that if you look at a ReQL GEOMETRY object in the Data Explorer or another driver, you will see that its internal representation is actually GeoJSON. However it will have an additional $reql_type$ field to identify it as a ReQL GEOMETRY object. r.geoJson adds that $reql_type$ field, and makes sure that the geometry is valid within the bounds given by ReQLs geometry implementation.

• geometry.to_geojson() : GEOMETRY -> OBJECT (JS: toGeojson) The opposite of r.geojson. Takes a ReQL GEOMETRY object and turns it into a valid GeoJSON object.
• r.point(lat, lon) : NUMBER, NUMBER -> GEOMETRY Constructs a ReQL GEOMETRY object of type point at the provided coordinates. See the "coordinates" section above for details.
• r.line(p1, p2, ..., pn) : GEOMETRY|ARRAY, ..., GEOMETRY|ARRAY -> GEOMETRY Constructs a ReQL GEOMETRY object of type line from the given vertices. The coordinates can be given in one of two ways:
  1. r.line([lat1, lon1], ..., [latn, lonn]), i.e. a sequence of arrays each having two numbers.
  2. r.line(r.point(lat1, lon1), ..., r.point(latn, lonn)), i.e. a sequence of GEOMETRY objects of type point.
• r.polygon(p1, p2, ..., pn) : GEOMETRY|ARRAY, ..., GEOMETRY|ARRAY -> GEOMETRY Same as r.line, but constructs a polygon instead that has the provided outer shell. The given shell does not have to be closed. If p1 != pn, r.polygon will close the shell implicitly by connecting pn to p1. Similar to r.line, each point can be given either as [lat, lon] or as a GEOMETRY object of type point. Note that r.polygon cannot construct polygons with holes in them. The sub can be used to create polygons with holes (see below).
• r.circle(center, radius) : GEOMETRY|ARRAY, NUMBER {num_vertices:NUMBER, geo_system:STRING, unit:STRING, fill:BOOL} -> GEOMETRY Constructs either a line or a polygon approximating a circle of radius around center. The approximation uses num_vertices (numVertices in JS) vertices, with 32 being the default. Those vertices are placed at a distance of radius around center, so to be precise the resulting polygon/line will have an exradius of radius. If fill is true (default), a polygon is constructed. Otherwise a line is constructed. The usual opt args for commands that deal with distances are supported. See "distances" under GENERAL.
• geometry.fill() : GEOMETRY -> GEOMETRY Takes a line and creates a polygon with the line as its shell. Behaves just as if the vertices of the line had been given to r.polygon.
• geometry1.distance(geometry2) : GEOMETRY, GEOMETRY {geo_system:STRING, unit:STRING} -> GEOMETRY Computes the distance between a point and another geometrical object, as described under "dinstances" in GENERAL. At least one out of geometry1 and geometry2 must be a point. If the other argument is a line or polygon, the projection of the point onto that line/polygon will be computed first, assuming a perfect sphere model. The distance between the point and its projection onto the line/polygon is then computed using the more precise ellipsoid model. If lines/polygons are huge compared to earth's radius and the distance between the point and the line/polygon is huge compared to earth's radius, the results of distance should be considered approximate, since the projection might be imprecise due to its simplified perfect-sphere assumption. The usual opt args for commands that deal with distances are supported.
• geometry1.intersects(geometry2) : GEOMETRY, GEOMETRY -> BOOL | Sequence, GEOMETRY -> Sequence Tests whether geometry1 intersects with geometry2, i.e. if they have any point(s) in common. When applied to a sequence, intersects works as a filter.
• geometry1.includes(geometry2) : GEOMETRY, GEOMETRY -> BOOL | Sequence, GEOMETRY -> Sequence Tests whether geometry2 is completely contained within geometry1. geometry1 must be a polygon, while geometry2 can be an arbitrary GEOMETRY object. When applied to a sequence, includes works as a filter.
• table.get_intersecting(geometry) : TABLE, GEOMETRY {index:STRING} -> StreamSelection (JS: getIntersecting) The index opt arg is mandatory and must correspond to a geospatial index. get_intersecting uses a geospatial index to find documents that intersect with geometry on the indexed field. It is semantically equivalent to table.filter(r.row("indexedField").intersects(geometry)). Note that the total number of results that get_intersecting can return is limited by the array size limit (default 100,000, can be changed through the opt arg array_limit I think). If it encounters more results, an error will be thrown.
• table.get_nearest(point) : TABLE, GEOMETRY|ARRAY {index:STRING, max_results:NUM, max_dist:NUM, geo_system:STRING, unit:STRING} -> ARRAY (JS: getNearest) The index opt arg is mandatory and must correspond to a geospatial index. get_nearest uses the given index to find documents in table that are close to point. point can be specified as either an array with latitude and longitude, or as a GEOMETRY of type point. The distance definition is equivalent to the one used by the distance term (see above). The result of get_nearest is an array of objects, where each object has the structure {dist: NUMBER, doc: OBJECT} where dist is the distance between point and the document in the specified unit (default: meters), and doc is the actual document from table. Results in the array are sorted in ascending order by their dist fields. A maximum of max_results (JS: maxResults, default 100) are returned, and a maximal distance of max_dist (JS: maxDist, default 100 km ~ 62 miles) is searched. The usual opt args for commands that deal with distances are supported.
• polygon1.polygon_sub(polygon2) : GEOMETRY, GEOMETRY -> GEOMETRY (JS: polygonSub) Both polygon1 and polygon2 must be polygons. polygon2 must only have a shell and no holes, and it must be completely contained in polygon1. Constructs a new polygon which is the same as polygon1, but with polygon2 cut out as a hole. This is the only way in ReQL to construct polygons with holes in them (apart from r.geojson)

[danielmewes]

Updated the last paragaph to reflect the introduction of the polygon_sub term to replace the overloaded sub.

[neumino]

I think this issue is fixed in 1.15.

The API docs, and the article have been written and have been reviewed.

I'm generating the docs for the python driver and the data explorer.

[chipotle]

Committed to v1.15 branch in commit 504197a. (Also see #517.)