issues with analytics dashboard

[1zun4secondary]

Hello. So I want to use these analytics for my API, but there a following issues:

• stops counting at 1000 requests(?)
• shows every request as endpoint (causes extreme long endpoint list)
• it should be possible to filter out 404 from all data (on my API it will cause success rate to stay at almost 0% and show many endpoints which do not even exist)

endless list of endpoints...

[tom-draper]

Great feedback! Thanks for raising these issues. I'll look into each of these when I'm next at my desk. Thanks!

[1zun4secondary]

For testing purposes I deployed it myself (https://github.com/CCBlueX/api-analytics/tree/deploy).

After deploying I had over 5k recorded requests within seconds. I don't think Supabase is the right choice for this kind of data, because it grows very fast. and if I would have to guess I think Supabase is restricting to 1000 results per query, which causes the dashboard to stay stuck after 1000 requests, but I did not have enough time to look into it.

Anyway storing each request is probably not the best idea, we need to combine them down to only the useful information that we require. It should also not store things like IP addresses but rather Location data like Country.

[tom-draper]

I had a feeling there was a better solution than one table row per request, it's just that each request needs a timestamp for the graphs. I thought about bucketing them into 5-10 minutes. IP addresses are stored to count unique visitors. I was looking into storing location but haven't found a way to store a GeoIP database locally with the backend serverless functions on Vercel so I'll have to host it separately or change hosting.

I've tried to minimise the size of a database row but I'm sure it could be better. The Supabase requests table stores: api_key uuid request_id int4 user_agent text method int2 created_at timestamptz path text status int2 response_time int2 framework int2 hostname text ip_address varchar

There's also a location varchar in there but it's currently not in use.

If the Supabase 1k query limit is true I'll have to find an alternative. I'll look into it further tonight.

Thanks so much for your help

[tom-draper]

The endpoints displayed in the dashboard have been capped at 50 and the styling has been updated to better display long endpoints. A button to remove requests with status 404 from the entire data set has been added to the top of the dashboard.

Supabase has a default limit of 1,000 rows per query, but this has now been raised to 15,000 which is still not ideal. Any higher and the max request body size for AWS serverless functions is exceeded. I'll look into ways to solve this problem.

[1zun4secondary]

Yup. I noticed it being raised to 15k

[tom-draper]

Closing this issue as all points raised have been solved.