developmentseed / tipg

Simple and Fast Geospatial OGC Features and Tiles API for PostGIS.
https://developmentseed.org/tipg/
MIT License
153 stars 23 forks source link

update search_path with user schemas #48

Closed vincentsarago closed 1 year ago

vincentsarago commented 1 year ago

closes #44

This PR tries to implement a solution based on https://github.com/developmentseed/tipg/issues/44#issuecomment-1486973429 but local tests shows that it doesn't work

# Launch database
docker-compose up database -d 

# Create a copy of `public.countries` in `myschema.countries`
psql postgresql://username:password@0.0.0.0:5439/postgis -c "CREATE SCHEMA IF NOT EXISTS myschema; SET schema 'myschema'; DROP TABLE IF EXISTS myschema.countries CASCADE; CREATE TABLE myschema.countries AS SELECT * FROM public.countries; SELECT count(*) FROM myschema.countries;"

# Start TiPG service
DATABASE_URL=postgresql://username:password@0.0.0.0:5439/postgis TIPG_DB_SCHEMAS='["myschema"]'  uvicorn tipg.main:app --port 8000 --reload

when function_schemas is set to [] using `TIPG_DB_FUNCTION_SCHEMAS='[]' then it works.

vincentsarago commented 1 year ago

🤔 tests pass locally but not in CI

I'll wait for a pre-review from @bitner before doing anything else 🙏 😄

vincentsarago commented 1 year ago

again after last commit, tests pass locally but I'm having hard time being able to reproduce the tests (it fails sometimes 🤷 )

I might update the fixtures to make sure we get 💯 reproductible tests

KoalaGeo commented 1 year ago

That's now partially working for me:

image

At the Items (http://tipg-2621-schemafix.kube-idev.bgslcdevops.test/collections/published.ukwm_iff_data_web/items & http://tipg-2621-schemafix.kube-idev.bgslcdevops.test/collections/published.ukwm_iff_data_web/items?f=geojsonseq) level I'm getting the below error. ?f=csv works fine.

[2023-03-30 21:28:08 +0000] [10] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/opt/bitnami/python/lib/python3.11/site-packages/anyio/streams/memory.py", line 94, in receive
    return self.receive_nowait()
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/base.py", line 78, in call_next
    message = await recv_stream.receive()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/anyio/streams/memory.py", line 114, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/bitnami/python/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 419, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette_cramjam/middleware.py", line 112, in __call__
    await responder(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette_cramjam/middleware.py", line 142, in __call__
    await self.app(scope, receive, self.send_with_compression)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/tipg/middleware.py", line 34, in dispatch
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/cors.py", line 84, in __call__
    await self.app(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/opt/bitnami/python/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/opt/bitnami/python/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/opt/bitnami/python/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/bitnami/python/lib/python3.11/site-packages/tipg/factory.py", line 954, in items
    request, orjson.dumps(data).decode(), template_name="items"
             ^^^^^^^^^^^^^^^^^^
TypeError: Type is not JSON serializable: decimal.Decimal
vincentsarago commented 1 year ago

@KoalaGeo let's open another issue for the decimal error 🙏

vincentsarago commented 1 year ago

🙏 thanks @bitner and @KoalaGeo