jovandeginste / workout-tracker

A workout tracking web application for personal use (or family, friends), geared towards running and other GPX-based activities
Other
958 stars 30 forks source link

WIP: perf: do not fetch all data for workouts to speed up rendering the workout list #25

Closed pixelspark closed 8 months ago

pixelspark commented 8 months ago

When loading the workout list, the app fetches basically the full workouts table including all GPX data. This seems a bit unnecessary and also is very slow. This PR fixes it partially by omitting certain fields. However, the Data field is still quite large (while the list only uses some statistics). Further work should probaby be done to split the Data field into a Details field (containing e.g. all the points) and a Statistics field containing the basic stats.

This change makes the query a full second faster on my data set (~400 workouts).

Before:

12:14PM WRN slow sql query [4.46089625s >= 100ms] app=workout-tracker version=local sha=local module=database slow_query=true query="SELECT * FROM `workouts` WHERE `workouts`.`user_id` = 1 AND `workouts`.`deleted_at` IS NULL ORDER BY date DESC" duration=4.46089625s rows=412 file=/Users/tommy/Repos/workout-tracker/pkg/database/user.go:200

After:

12:40PM WRN slow sql query [3.434046834s >= 100ms] app=workout-tracker version=local sha=local module=database slow_query=true query="SELECT `workouts`.`id`,`workouts`.`created_at`,`workouts`.`updated_at`,`workouts`.`deleted_at`,`workouts`.`name`,`workouts`.`date`,`workouts`.`user_id`,`workouts`.`dirty`,`workouts`.`notes`,`workouts`.`type`,`workouts`.`data`,`workouts`.`filename` FROM `workouts` WHERE `workouts`.`user_id` = 1 AND `workouts`.`deleted_at` IS NULL ORDER BY date DESC" duration=3.434046834s rows=412 file=/Users/tommy/Repos/workout-tracker/pkg/database/user.go:210
jovandeginste commented 8 months ago

I think we should move those other fields to their separate tables. Then it's a matter of (not) preloading those associations.

I was working on this earlier, but didn't get it right yet.

jovandeginste commented 8 months ago

Check this PR: #26

pixelspark commented 8 months ago

Yes, that looks nice :+1:

It appears Data can also get quite heavy, you might want to move that out of the way as well (and only have some basic statistics in the main workouts table derived from it). I am not sure if gorm supports it but you might be able to utilize SQLite JSON functionality (possibly with virtual columns and even indexes on that?) to implement this. (Then again you also want to support other databases)

jovandeginste commented 8 months ago

The plan was to get data out of the way, but that one is a little harder; I wanted to test with the easier column first 😄