search

LINBIT / csync2

file synchronization tool using librsync and current state databases
GNU General Public License v2.0
145 stars 39 forks source link

will csync2 support 1TB of data bidirectional replication. #15

Closed raghumuppa closed 4 years ago

raghumuppa commented 5 years ago

i have small doubt i wants to clarify it, can csync2 replicate 1TB of data in bidirectional all the time i mean whenever changes happen on node1 or on node2 data has to be replicated to and fro, just like mirroring and what are the limitations of csync2 what is max size of a file it can replicate.

csync2 can replicate huge files like 5Gb iso images.

Thanks & Regards

lge commented 4 years ago

Short answer: yes.

Longer answer: probably not for your use case, I get the feeling you misunderstand what csync2 is intended to do.

If what you desire is live-replication of some continuously changing file (e.g. in-use VM image or database tablespace or something like that), csync2 is the wrong tool. If you could do whatever you want by using "rsync in a loop", you can use csync2 (and gain some features, and lose others). If not, then csync2 won't help either.

Conceptually, csync2 is "just" rsync on drugs. It also is push only. Csync2 is working in several phases. Simplified:

1) init: record current stat() information in some database 2) check: compare current stat() information with that database, flag entries as "in need of update", and record the now current information. 3) update: try to push flagged changes to the peers, using librsync.

After changes have been successfully pushed to some peer, their "need update" flag is cleared for that peer. "conflict" detection happens on the receiving side by double checking if current and recorded stat() information still match. So yes, you can use csync2 to "replicate" one single huge file, or millions of smaller files, or billions of huge files, whatever. Just like you can rsync them.

"check" time will go up with number of necessary stat calls (and the latency for those on your IO backend), "update" time will go up for the single file with the size of the file, the size of the actual change in that file, and how nicely that can be "squeezed" by librsync.

Just use the right tool for the job. So if what you reall want is live replication, use DRBD. If you want snapshot shipping, use something designed for that (ZFS comes to mind). If you want something else, use something else. And if you want your cake and eat it, too, good luck with that ;-)

What do you want?

raghumuppa commented 4 years ago

Thanks for the clarification