FrigadeHQ / trench

Trench — Open-Source Analytics Infrastructure. A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events, page views. Easily build product analytics dashboards, LLM RAGs, observability platforms, or any other analytics product.
https://trench.dev
MIT License
1.53k stars 54 forks source link

Get Events API null total #13

Closed esadek closed 3 months ago

esadek commented 4 months ago

I am working through the quickstart and am getting total": null when using the Get Events API. The docs say "If null, the total is unknown" but I don't think this should be the case since ClickHouse contains only one event.

> curl -i -X GET \ 
   -H "Authorization: Bearer private-d613be4e-di03-4b02-9058-70aa4j04ff28" \
   'http://localhost:4000/events?event=ConnectedAccount'

HTTP/1.1 200 OK
access-control-allow-origin: *
access-control-allow-credentials: true
access-control-expose-headers: Authorization
content-type: application/json; charset=utf-8
content-length: 586
Date: Sat, 02 Nov 2024 00:48:52 GMT
Connection: keep-alive
Keep-Alive: timeout=72

{"results":[{"uuid":"094ea003-2e9c-41e2-9ad6-ed49bfa10d1a","type":"track","event":"ConnectedAccount","userId":"550e8400-e29b-41d4-a716-446655440000","properties":{"totalAccounts":4,"country":"Denmark"},"timestamp":"2024-11-02T00:15:25.000Z","parsedAt":"2024-11-02T00:15:31.211Z"},{"uuid":"d2c6194a-522b-48e6-b0b4-1aa8f652f253","type":"track","event":"ConnectedAccount","userId":"550e8400-e29b-41d4-a716-446655440000","properties":{"totalAccounts":2,"country":"Germany"},"timestamp":"2024-11-02T00:22:26.000Z","parsedAt":"2024-11-02T00:22:31.166Z"}],"limit":1000,"offset":0,"total":null}
christianmat commented 4 months ago

This looks like a bug indeed. Feel free to open a PR. Given that it requires a secondary query to ClickHouse, we may want to have the total only be shown if it’s needed and add a flag / query param for it.

esadek commented 4 months ago

Can't null just be changed to results.length?

https://github.com/FrigadeHQ/trench/blob/0e243b4915d73d83433864e12370b24c88a3ceef/apps/trench/src/events/events.dao.ts#L93-L98

christianmat commented 4 months ago

The total is meant to reflect the total amount of results to be used with pagination (e.g. calculating total number of pages to show an end user).

tuminzee commented 4 months ago

@christianmat I would like to open a PR for this

christianmat commented 4 months ago

@tuminzee go for it :)

tuminzee commented 3 months ago
const totalQuery = `SELECT count() FROM events ${whereClause}`
const total = await this.clickhouse.queryResults(totalQuery, workspace.databaseName)
total: total?.[0]?.count ?? 0,