octue / octue-sdk-python

The python SDK for @Octue services and digital twins.
https://octue.com
Other
10 stars 4 forks source link

Stream service events into BigQuery #620

Closed cortadocodes closed 5 months ago

cortadocodes commented 12 months ago

Feature request

IMPORTANT: We need to test this before merging the breaking services communication change in case we need to change the format of event attributes (see #603).

Use Case

This will allow advanced querying of events at a lower storage cost and database server cost (as fewer concurrent connections to the database server will be needed e.g. by django-twined's service usage events.

Current state

Events are stored by django-twined in a postgresql database and the database server has to be upgraded to cope with many questions being asked at once.

More info