search

graphite-project / carbonate

Utilities for managing graphite clusters
MIT License
516 stars 80 forks source link

Intermittent rsync errors #61

Closed bwilder closed 7 years ago

bwilder commented 8 years ago

Hi, I'm getting intermittent rsync errors while running a script based on the 'Rebalance a cluster' example. The errors look like below:

WARNING:root:Failed to copy /var/lib/graphite/whisper/myprdbox1/disk/disk/mapper/rootvg-rootlv/avail.wsp! [Errno 2] No such file or directory: '/tmp/tmpgzUPfI/myprdbox1/disk/disk/mapper/rootvg-rootlv/avail.wsp'

Rerunning the script rarely yields the same errors. Sometimes it finishes cleanly, throwing no errors at all. Googling and troubleshooting have yet to yield any satisfying results. Do you have any insight into what might cause this behavior? Would this kind of thing happen if attempting to rsync a database that's currently being written to?

If there's specific data I can provide to help narrow the scope, let me know.

Thanks!

adubkov commented 8 years ago

I have same issue while syncing from old stand alone box to new cluster box.

dinapappor commented 7 years ago

I am having the exact same issue as @blacked . Were there ever a fix for this?

dinapappor commented 7 years ago

Ok, this was due to not having rsync on original box.

deniszh commented 7 years ago

Just additional note - if you use locking you may find libflockit useful to make rsync obey to the locks. Check this blog, flockit repo and corresponding issue on carbonate project.