NASA-IMPACT / veda-backend

Backend services for VEDA
Other
13 stars 5 forks source link

fix: support automatic setting of search_path #317

Closed moradology closed 9 months ago

moradology commented 9 months ago

This is a workaround for the error being encountered by titiler-pgstac's use of command line arguments. Application can be set via query paramaters and search_path is set by automatically executing SET search_path upon acquisition of connection

moradology commented 9 months ago

Setting to draft for now to avoid accidental merger. Not sure about the test failures - so far as I can tell this follows expectations for postgres DSN strings

vincentsarago commented 9 months ago

FYI, another solution (once https://github.com/stac-utils/titiler-pgstac/pull/155 get's published) would have been to use a custom mosaic backend


from titiler.pgstac import mosaic

@attr.s
class PGSTACBackend(mosaic.PGSTACBackend)
    @cached(  # type: ignore
        TTLCache(maxsize=cache_config.maxsize, ttl=cache_config.ttl),
        key=lambda self, geom, **kwargs: hashkey(self.input, str(geom), **kwargs),
    )
    @retry(
        tries=retry_config.retry,
        delay=retry_config.delay,
        exceptions=(
            pgErrors.OperationalError,
            pgErrors.InterfaceError,
        ),
    )
    def get_assets(
        self,
        geom: Geometry,
        fields: Optional[Dict[str, Any]] = None,
        scan_limit: Optional[int] = None,
        items_limit: Optional[int] = None,
        time_limit: Optional[int] = None,
        exitwhenfull: Optional[bool] = None,
        skipcovered: Optional[bool] = None,
    ) -> List[Dict]:
        """Find assets."""
        fields = fields or {
            "include": ["assets", "id", "bbox", "collection"],
        }

        scan_limit = scan_limit or 10000
        items_limit = items_limit or 100
        time_limit = time_limit or 5
        exitwhenfull = True if exitwhenfull is None else exitwhenfull
        skipcovered = True if skipcovered is None else skipcovered

        with self.pool.connection() as conn:
            with conn.cursor() as cursor:
                try:
                    cursor.execute("SET search_path TO pgstac, public")
                    cursor.execute(
                        "SELECT * FROM geojsonsearch(%s, %s, %s, %s, %s, %s, %s, %s);",
                        (
                            geom.model_dump_json(exclude_none=True),
                            self.input,
                            json.dumps(fields),
                            scan_limit,
                            items_limit,
                            f"{time_limit} seconds",
                            exitwhenfull,
                            skipcovered,
                        ),
                    )
                    resp = cursor.fetchone()[0]

                except pgErrors.RaiseException as e:
                    # Catch Invalid SearchId and raise specific Error
                    if f"Search with Query Hash {self.input} Not Found" in str(e):
                        raise MosaicNotFoundError(
                            f"SearchId `{self.input}` not found"
                        ) from e
                    else:
                        raise e

        return resp.get("features", [])
moradology commented 9 months ago

Looks to me like bumping titiler is the path forward here; closing this for now