apache / incubator-baremaps

Create custom vector tiles from OpenStreetMap and other data sources with Postgis and Java.
baremaps.apache.org
Apache License 2.0
491 stars 56 forks source link

Create tiles with data from multiple databases #845

Open bchapuis opened 3 months ago

bchapuis commented 3 months ago

Tilesets currently connect to a single database. It would be nice if a tileset could span and connect to several databases.

One possible implementation would consist in specifying the data source at the layer level in the tileset. This way the TileStore could group layers by data source, fetch the layers by data source, and concatenate the layers in the TileStore. This is probably the easiest way to implement this feature, however, it means that each layer query will always involve only one database.

{
"datasources":[
 {
   "id": "a"
   "jdbcUrl": "jdbc:postgresql://a:5432/a",
   "poolSize": 8
 },
 {
   "id": "b"
   "jdbcUrl": "jdbc:postgresql://b:5432/b",
   "poolSize": 8
 }
],
"vector_layers": [
    {
      "id": "layer-a",
      "datasource": "a"
      "queries": [
        {
          "minzoom": 12,
          "maxzoom": 20,
          "sql": "SELECT id, tags, geom FROM table"
        },
      ]
    },
   {
      "id": "layer-b",
      "datasource": "b"
      "queries": [
        {
          "minzoom": 12,
          "maxzoom": 20,
          "sql": "SELECT id, tags, geom FROM table"
        },
      ]
    },
]

Another possibile implementation may be to rely on apache calcite to provide a uniform access to several databases. This solution would allow to execute queries that span over multiple databases. Personally, I'm a bit more interested in this alternative, as I currently seek to improve the compatibility of calcite with postgis. Unfortunately, it is probably not practical in a short to medium time frame.

https://github.com/apache/calcite/pull/3668

I suggest we go for the first option, keeping in mind that apache calcite may be a better solution in the long term. @julsbreakdown do not hesitate to comment and to provide feedback.

bchapuis commented 3 months ago

We should probably find a way to set a default datasource.