search

rocicorp / replicache

Realtime Sync for Any Backend Stack
https://doc.replicache.dev
1.04k stars 37 forks source link

We need to stabilize the perf benchmarks somehow #216

Closed aboodman closed 3 years ago

aboodman commented 3 years ago

This is close to being useful:

https://replicache-sdk-js-perf-git-perf-data.rocicorp.vercel.app/

... but not useful. See for example the dip in populate perf around 63c44cc24cdeaa40053b533625f3bf0b380ed985. Is it real? If so which commit? Hard to tell.

@phritz it seems like it should not be hard to get this to run on either an dedicated instance in ec2 or at least isolated? I would like to add this to near term work. wdyt?

phritz commented 3 years ago

@phritz it seems like it should not be hard to get this to run on either an dedicated instance in ec2 or at least isolated? I would like to add this to near term work. wdyt?

Definitely seems important to have reliable metrics for the things that matter most, so yes, let's do it.

phritz commented 3 years ago

Something we should address: that we dont get fine-grain commit results for repc changes. We should definitely fix this so we don't have to go digging into releases to find when a thing happened. Maybe a smoke perf test on the repc side to guide the search or test-releasing every commit in repc so we can run the benchmarkes on the js side.

phritz commented 3 years ago

moved populate degredation issue, which is separate, to https://github.com/rocicorp/replicache-sdk-js/issues/218

arv commented 3 years ago

Closing for now.

We are running the tests on our own EC2 instance which should be more stable.