onemoredata / bagger

Massive log storage in PostgreSQL
BSD 2-Clause "Simplified" License
9 stars 0 forks source link

Insert Routing Trigger (with stub functions) #71

Open einhverfr opened 8 months ago

einhverfr commented 8 months ago

After discussions with Felix, we have decided to break #14 into more reviewable tickets and close that one.

In this one we will have a simple routing trigger which has stub functions for other related tasks such as table creation, cache plan management etc. This one will be simple and is not expected to solve anything other than:

  1. Route the inserts into the correct table
  2. Detect if the table doesn't exist and then direct to a stub function

Plan caching, table creation, and bad data handling will all come later.

einhverfr commented 1 month ago

So looking at this, what I think needs to be done is:

  1. Create a state memory context and a cache memory context if not exist
  2. Query and store the dimension info in a single linked list if not exist
  3. Generate the name of the table from the dimensions and JSON
  4. Check if we have a cached plan and use it if we have.
  5. If we don't hjave a cached plan, write one, and then use it.

It seems to me we can probably use Postgres datetime logic internally.

How does this sound?

einhverfr commented 1 month ago

We will need to store the following in the state context: generation information and dimension information