Qovery / Replibyte

Seed your development database with real data ⚡️
https://www.replibyte.com
GNU General Public License v3.0
4.17k stars 129 forks source link

Slow local dump restore #116

Open evoxmusic opened 2 years ago

evoxmusic commented 2 years ago

I was making some tests, and using replibyte to restore in local is quite slow.

~/D/replibyte $ replibyte -c strapi-v4.yaml dump list
⠦
 name                 | size  | when        | compressed | encrypted
----------------------+-------+-------------+------------+-----------
 dump-1652739105408   | 11 kB | 8 hours ago | true       | false
⠖
~/D/replibyte $ time replibyte -c strapi-v4.yaml dump restore local --image postgresql -v latest --port 8989
To connect to your PostgreSQL instance, use the following connection string:
> postgres://postgres:password@localhost:8989/postgres
Waiting for Ctrl-C to stop the container
^Ccontainer stopped!

________________________________________________________
Executed in   25.14 secs      fish           external
   usr time  348.08 millis    0.06 millis  348.02 millis
   sys time  191.67 millis    1.14 millis  190.53 millis

It takes ~25s to run replibyte -c strapi-v4.yaml dump restore local --image postgresql -v latest --port 8989 with a dump of 11kB.

If I run the command with -o it is far faster

~/D/replibyte $ time replibyte -c strapi-v4.yaml dump restore local -v latest -o > /dev/null

________________________________________________________
Executed in    1.78 secs      fish           external
   usr time  117.02 millis    0.08 millis  116.94 millis
   sys time   66.82 millis    2.61 millis   64.20 millis

It will be great to see how we can improve the docker starting time then :)

fabriceclementz commented 2 years ago

As an enhancement, we could start by fixing this workaround I’ve made while developing the first version of the restore local command https://github.com/Qovery/Replibyte/blob/bd0604cebab381aa0378ccd83af30d3d97101b0b/replibyte/src/destination/docker.rs#L49

A better approach is to poll the container status to wait until he’s up and running.

evoxmusic commented 2 years ago

ahhhhh yes indeed 😂