tigrisdata-archive / tigris-client-go

Tigris data-platform Golang client
https://www.tigrisdata.com/docs/sdkstools/golang/
Apache License 2.0
17 stars 5 forks source link

Implement client side in memory joins #280

Open efirs opened 1 year ago

efirs commented 1 year ago

To simplify explanation the proposal below uses this specific example models:

type User struct {
  ID uuid.UUID `json:"id"`
  Name string `json:"name"`
}

type Order struct {
  ID uuid.UUID 
  UserID uuid.UUID `json:"user_id"` // references User's ID field
  Quantity int
}

The API would allow to join two tables based on field equality:

userOrders, err := tigris.GetJoin[User, Order](db, "id", "user_id")

Now results can be read by:

it, err := userOrders.Read(ctx, filter.Eq("name"))

where filter is applied to the left table.

Results could be iterated as:

var user User
var orders []Order

for it.Next(&user, &orders) {
  fmt.Printf("user=%v orders=%v\n", user, orders)
}

Implementation details:

First request is issued to the left table with filter provided to Read API. Result is read into memory and request is prepared for the right table. Which will have the following filter filter.Or(filter.Eq("user_id", {value of id field}), ....).

Result from the first query is put in the map with ID as the key, while reading the result from second query we append it to the corresponding map bucket.

So as merge is done in the memory, this approach is not effective for very large results.