maplibre / martin

Blazing fast and lightweight PostGIS, MBtiles and PMtiles tile server, tile generation, and mbtiles tooling.
https://martin.maplibre.org
Apache License 2.0
2.33k stars 215 forks source link

/catalog doesn't load pg function; tables are working fine #1473

Closed Zia- closed 3 months ago

Zia- commented 3 months ago

Hi there,

I'm struggling with loading a custom pg function to view some random points. The table public.randompoints has geom geometry column. The /catalog endpoint shows all tables nicely, however I was expecting _function_zxyquery as well to appear there. Directly hitting _/function_zxyquery endpoint says _Source function_zxyquery does not exist

This is the function:

CREATE OR REPLACE
    FUNCTION function_zxy_query(z integer, x integer, y integer)
    RETURNS bytea AS $$
DECLARE
  mvt bytea;
BEGIN
  SELECT INTO mvt ST_AsMVT(tile, 'function_zxy_query', 4096, 'geom') FROM (
    SELECT
      ST_AsMVTGeom(
          ST_Transform(ST_CurveToLine(geom), 3857),
          ST_TileEnvelope(z, x, y),
          4096, 64, true) AS geom
    FROM public.randompoints
    WHERE geom && ST_Transform(ST_TileEnvelope(z, x, y), 4326)
  ) as tile WHERE geom IS NOT NULL;

  RETURN mvt;
END
$$ LANGUAGE plpgsql IMMUTABLE STRICT PARALLEL SAFE;

Exactly how it's defined here, except my table name ie public.randompoints

And below is what the config.yaml looks like:

keep_alive: 75
listen_addresses: '0.0.0.0:3000'
base_path: /tiles
worker_processes: 3
cache_size_mb: 1024
preferred_encoding: gzip
web_ui: enableforall
postgres:
  - connection_string: '<conn_string>'
    default_srid: 4326
    pool_size: 20
    max_feature_count: 10000
    auto_bounds: skip
    auto_publish:
      from_schemas:
        - public
      tables:
        source_id_format: '{schema}.{table}.{column}'
    functions:
      function_source_id:
        schema: public
        function: function_zxy_query
        minzoom: 0
        maxzoom: 30
        bounds: [ -180.0, -90.0, 180.0, 90.0 ]

Would highly appreciate if someone can point me where things are broken. Thanks!

nyurik commented 3 months ago

Please include martin version, and also run this query - see what results it shows. Lastly, do you see anything relevant in the logs? See here.

Zia- commented 3 months ago

Most likely it's this version https://github.com/maplibre/martin/pkgs/container/martin/235708266?tag=v0.14.2. I didn't add any tag before pushing it to our company's internal harbor (my bad!). But since all this I did like 2-3 weeks ago, it should be 0.14.2.

So just to give you some context here, I'm running Martin on K8s. This means I have to add a few certs and proxies to Martin's image to bypass all firewalls before deploying it. Kindly let me know if more details on k8s part is needed.

The SQL gave this: image

Below is what the pod log looks like (I've masked what's confidential). The thing is I see tables being reported as Discovered but not the pg function:

[2024-08-16T14:18:59Z INFO  martin] Starting Martin v0.15.0
[2024-08-16T14:18:59Z INFO  martin] Using /etc/martin/martin.yaml
[2024-08-16T14:18:59Z INFO  martin::config] Initializing main cache with maximum size 1073741824B
[2024-08-16T14:18:59Z INFO  martin::pg::pool] Connecting with SSL support: Config { user: Some("<>"), password: Some(_), dbname: Some("<>"), options: None, application_name: Some("Martin v0.15.0 - pid=1"), ssl_mode: Prefer, host: [Tcp("<>")], hostaddr: [], port: [5432], connect_timeout: None, tcp_user_timeout: None, keepalives: true, keepalives_idle: 7200s, keepalives_interval: None, keepalives_retries: None, target_session_attrs: Any, channel_binding: Prefer }
[2024-08-16T14:18:59Z INFO  martin::pg::pool] Connecting with SSL support: Config { user: Some("<>"), password: Some(_), dbname: Some("<>"), options: None, application_name: Some("Martin v0.15.0 - pid=1"), ssl_mode: Prefer, host: [Tcp("<>")], hostaddr: [], port: [5432], connect_timeout: None, tcp_user_timeout: None, keepalives: true, keepalives_idle: 7200s, keepalives_interval: None, keepalives_retries: None, target_session_attrs: Any, channel_binding: Prefer }
[2024-08-16T14:18:59Z INFO  martin::pg::pool] Connecting with SSL support: Config { user: Some("<>"), password: Some(_), dbname: Some("<>"), options: None, application_name: Some("Martin v0.15.0 - pid=1"), ssl_mode: Prefer, host: [Tcp("<>")], hostaddr: [], port: [5432], connect_timeout: None, tcp_user_timeout: None, keepalives: true, keepalives_idle: 7200s, keepalives_interval: None, keepalives_retries: None, target_session_attrs: Any, channel_binding: Prefer }
[2024-08-16T14:19:02Z WARN  martin::pg::query_tables] Table public.geometries has no spatial index on column geom
[2024-08-16T14:19:02Z WARN  martin::pg::query_tables] Table <>.tb_aois has no spatial index on column footprint
[2024-08-16T14:19:02Z WARN  martin::pg::query_tables] Table <>.tb_aois_old has no spatial index on column footprint
[2024-08-16T14:19:02Z WARN  martin::pg::query_tables] Table <>.tb_catalogue has no spatial index on column footprint
[2024-08-16T14:19:02Z WARN  martin::pg::query_tables] Table <>.tb_catalogue_old has no spatial index on column search_geometry
[2024-08-16T14:19:02Z INFO  martin::pg::builder] Auto-publishing tables in schemas [public, <>] as '{schema}.{table}.{column}' sources
[2024-08-16T14:19:02Z INFO  martin::pg::config_table] Table public.geometries.geom has SRID=0, using provided default SRID=4326
[2024-08-16T14:19:02Z INFO  martin::pg::builder] Discovered source public.geometries.geom from table public.geometries with geom column (GEOMETRY, SRID=4326)
Zia- commented 3 months ago

MY BAD! I was referring to a wrong config.yaml from deployment.yaml in k8s. It's horrible. All working just fine now.