Replace application by new consolidated Features/MVT service?

[j08lue]

From @vincentsarago at https://github.com/NASA-IMPACT/veda-features-api/issues/8#issuecomment-1288676134

FYI: We are moving away from tifeatures/timvt to https://github.com/developmentseed/tipg (empty for now) which will have both mvt and features endpoints.

Great. Would you recommend that we replace our app here with the new service, to get both, the Features and MVT endpoints out of the box?

And when do you expect the new service to be ready?

[vincentsarago]

Great. Would you recommend that we replace our app here with the new service, to get both, the Features and MVT endpoints out of the box?

yes

And when do you expect the new service to be ready?

end of the year 🤞 (cc @bitner )

[smohiudd]

@vincentsarago has there been discussion around db schema for vector features? I didn't see anything in https://github.com/developmentseed/eoAPI or tipg. Or does each collection have its own table?

[vincentsarago]

Or does each collection have its own table

in tipg/tifeatures/timvt each collection is a table or a function (functions are not yet supported in tipg).

[smohiudd]

I added a branch with tipg (https://github.com/NASA-IMPACT/veda-features-api/tree/sm/feature-tipg) but having troubling getting the collection temporal extent because it's not picking up the datetime col in the fireline dataset

[smohiudd]

The datetime column in the fireline dataset was not cast in the correct datatype after importing to postgis. After checking with @vincentsarago, confirmed that it must be TIMESTAMP. Cast in the correct datatype using:

ALTER TABLE fireline                                                                   
ALTER COLUMN t TYPE TIMESTAMP USING (t::TIMESTAMP);

This will probably lead to another discussion about how to deal with datetime data during ingestion and validation. Something to revisit for vector ingestion work.

[smohiudd]

I deployed an updated stack tifeatures-timvt-dev (https://pzh7fedpc0.execute-api.us-west-2.amazonaws.com/) that uses tipg instead of tifeatures. Everything works ok under local development however after deploying with cdk, getting an internal server error:

[ERROR] UndefinedFunctionError: operator does not exist: text[] || name
HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/var/lang/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/var/task/handler.py", line 41, in <module>
    loop.run_until_complete(app.router.startup())
  File "/var/lang/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/tmp/pip-target-_ffz4oxd/lib/python/starlette/routing.py", line 648, in startup
  File "/var/task/handler.py", line 28, in startup_event
    await register_collection_catalog(app)
  File "/tmp/pip-target-_ffz4oxd/lib/python/tipg/db.py", line 46, in register_collection_catalog
  File "/tmp/pip-target-_ffz4oxd/lib/python/tipg/dbmodel.py", line 1032, in get_collection_index
  File "/tmp/pip-target-_ffz4oxd/lib/python/buildpg/asyncpg.py", line 64, in fetch_b
  File "/tmp/pip-target-_ffz4oxd/lib/python/asyncpg/connection.py", line 620, in fetch
  File "/tmp/pip-target-_ffz4oxd/lib/python/asyncpg/connection.py", line 1658, in _execute
  File "/tmp/pip-target-_ffz4oxd/lib/python/asyncpg/connection.py", line 1683, in __execute
  File "/tmp/pip-target-_ffz4oxd/lib/python/asyncpg/connection.py", line 1710, in _do_execute
  File "/tmp/pip-target-_ffz4oxd/lib/python/asyncpg/connection.py", line 397, in _get_statement
  File "asyncpg/protocol/protocol.pyx", line 168, in prepare
    return await waiter

Looks like its related to this query in tipg. I wasn't getting this same server error with the tifeatures deployment

[smohiudd]

^ Was solved by upgrading to Postgres 14 from 13 in the cdk stack. The link above should work now.

[j08lue]

Can this be closed, @smohiudd?

[smohiudd]

Yes this can be closed.