moigagoo / norm

A Nim ORM for SQLite and Postgres
https://norm.nim.town
MIT License
378 stars 34 forks source link

Added bulk update with single query #189

Closed thisago closed 1 year ago

thisago commented 1 year ago

For now, just SQLite module is updating with single query, soon I'll implement in Postgres

Now I added bulk update for SQLite and PostgreSQL, The tests are running well

Closes #188

thisago commented 1 year ago

I think that I fixed all the requested changes in the last commit:

thisago commented 1 year ago

I guess it's a WIP for now, amirite?

I'm already using it and looks good for me, the requested changes was done

thisago commented 1 year ago

but if you'd be interested to adding benchmarks that illustrate the performance boost that would be great.

haha, ok, there's a better way to do the benchmark or just run it with time?

moigagoo commented 1 year ago

but if you'd be interested to adding benchmarks that illustrate the performance boost that would be great.

haha, ok, there's a better way to do the benchmark or just run it with time?

I was planning to use benchy for it.

The way I see it:

  1. Create benchmarks folder.
  2. Add benchy to norm.nimble.
  3. Add bench task to norm.nimble.
  4. Add a benchmark that updates 1000 rows
thisago commented 1 year ago

So, the old implementation was replaced, I will put a copy of the old implementation in the benchmark file just for it, ok?

thisago commented 1 year ago
  1. Add a benchmark that updates 1000 rows

1k of rows is too much to make multiple tests like benchy does... I'll limit to just one test ok? or you prefer to reduce rows, like 100 to repeat 10 times?

thisago commented 1 year ago

I ran 2 times the benchmark with 1k of rows and the result is very satisfactory: (for sqlite)

1th

min time      avg time       std dv  runs  name
   133.105 ms    133.105 ms  ±0.000  x1    Bulk update
139891.469 ms 139891.469 ms  ±0.000  x1    Update each row

2th

min time      avg time       std dv  runs  name
   129.890 ms    129.890 ms  ±0.000  x1    Bulk update
138162.341 ms 138162.341 ms  ±0.000  x1    Update each row
thisago commented 1 year ago

I updated the benchmark code to generate the DB just one time and copy into another file to modify.
I added also an indicator of DB generation progress, it shows a dot for every added row.

Also, an important thing to add next is bulk insert because inserting 1k of simple rows are too slow...

Another benchmark result:

min time      avg time       std dv  runs name
   182.585 ms    182.585 ms  ±0.000  x1   Bulk update
174604.512 ms 174604.512 ms  ±0.000  x1   Update each row
moigagoo commented 1 year ago

I'll do some minor fixes myself later and cut a new release.

Thanks for the great work!