search

DGKSK8LIFE / redisql

SQL to Redis caching made easy
https://dev.to/tarekali/how-i-solved-mysql-to-redis-caching-with-go-3lf7
MIT License
63 stars 14 forks source link

[BUG] CopyToString is over 100% slower than other methods #45

Open DGKSK8LIFE opened 3 years ago

DGKSK8LIFE commented 3 years ago

Expected Behavior

Performance in CopyToString should be comparable to that of CopyToList and CopyToHash.

Current Behavior

CopyToString is over 100% slower than the aforementioned two.

Possible Solution

Because of Redis' data structures, CopyToString (strings) has to write one key per field, compared to CopyToList and CopyToHash that both write one key per row. This is not necessarily something that can be "quick fixed", but with other performance optimizations, especially transactions and pipelining, performance should improve drastically.

Steps to Reproduce

  1. Run test cases with ~10k rows or so and your preferred DB.
  2. View output: 
    ➜  redisql git:(master) go test -v -db mysql -rows 10000 
Preparing Test...
=== RUN   TestCopyToString
--- PASS: TestCopyToString (8.43s)
=== RUN   TestCopyToList
--- PASS: TestCopyToList (0.75s)
=== RUN   TestCopyToHash
--- PASS: TestCopyToHash (0.92s)
PASS
ok      github.com/DGKSK8LIFE/redisql   17.995s
Fajrinmk commented 3 years ago

Hi @DGKSK8LIFE , have you solved this yet? I'd like to try to solve this

DGKSK8LIFE commented 3 years ago

I'm working on one possible solution, but the bug is not definitively resolved. Definitely want to see your solution so go ahead and work on this, ty. @Fajrinmk