whatyouhide / redix

Fast, pipelined, resilient Redis driver for Elixir. 🛍
http://hexdocs.pm/redix
MIT License
1.1k stars 122 forks source link

Slow requests for keys that hold large data (~70MB) #246

Closed saneery closed 1 year ago

saneery commented 1 year ago

I noticed that Redix is slow on large data, compared to eredis. Benchmarking results:

Name                      ips        average  deviation         median         99th %
eredis GET 70MB          0.62         1.61 s    ±18.19%         1.52 s         2.69 s
redix GET 70MB           0.24         4.18 s     ±5.54%         4.24 s         4.43 s

Comparison: 
eredis GET 70MB          0.62
redix GET 70MB           0.24 - 2.59x slower +2.57 s

Benchmarking script:

Benchee.run(
  %{
    "redix GET 70MB" => fn {_, redix} ->
      {:ok, _} = Redix.command(redix, ["GET", "keytest"])
    end,
    "eredis GET 70MB" => fn {eredis, _} ->
      {:ok, _} = :eredis.q(eredis, ["GET", "keytest"])
    end,
  },
  before_scenario: fn _ ->
    {:ok, eredis} = :eredis.start_link('127.0.0.1', 6379, 1)
    {:ok, redix} = Redix.start_link(host: "127.0.0.1", port: 6379, database: 1)

    large_data = :crypto.strong_rand_bytes(70000000)
    {:ok, _} = Redix.command(redix, ["SET", "keytest", large_data])
    {eredis, redix}
  end,
  time: 30
)
whatyouhide commented 1 year ago

Great catch! Should be fixed by https://github.com/whatyouhide/redix/pull/247.

saneery commented 1 year ago

Thanks a lot! Looking forward to the release in the hex.pm

whatyouhide commented 1 year ago

I released v1.2.2. 👍