welaika / wordmove

Multi-stage command line deploy/mirroring and task runner for Wordpress
https://wptools.it/wordmove
MIT License
1.87k stars 167 forks source link

Wordmove deleted a bunch of non-wordpress files from server #271

Closed paperrobots closed 6 years ago

paperrobots commented 8 years ago

Is there a setting for this? If not there should be.

I thought I was just updating my WP Core files and it removed all other files in the directory that weren't Wordpress. I had no idea it would do this and I did it on a client server, lost all their other files on their server. They didn't have a backup in place. I had never encountered this issue before otherwise I'd have backed up the other files myself. I only backed up Wordpress unfortunately.

alessandro-fazzi commented 8 years ago

OH MY GOD.

Belive me @paperrobots: I know that feeling.

Unfortunately Wordmove works just like that: it mirrors folders back and forth. No mercy. In detail wordmove relies on rsync --delete and lftp --mirror commands.

You have a way to use it leaving polluting files alone: using the exclude section inside Movefile.

That is a really static way to handle this kind of situation, but

  1. we think there is no other sane way
  2. we think the problem are polluted folder in production

I can see two action paths regard your issue:

  1. Pumping up a really big warning somewhere in the README
  2. Writing a PR adding an option to disable the mirroring flags (command line? Movefile? No idea. Glad to discuss inside a PR eventually)
alessandro-fazzi commented 8 years ago

@paperrobots (or anybody else interested) any thought/opinion you'd like to share?

paperrobots commented 8 years ago

It was just confusing when I'm pushing a flag for Wordpress I'm assuming it's just doing all the Wordpress files besides wp-content.

I agree file pollution in directories (especially top level) is awful, but sometimes out of our control between different clients. A statement about it in the readme could be good though!

alessandro-fazzi commented 8 years ago

Yep, I'm with you. We always used clean and dedicated remote directories, so we never faced the problem. But I agree sometimes someone hasn't full control over things...

Thinking about the «just doing all the Wordpress files» i can't figure how we should be aware about what they actually are: they could be specific to you workflow, to the WP version used and so on.

I'll think where to insert a clear statement about the expected behaviour. If you have suggestions or if you'd like to do a PR you're welcome.

ghost commented 8 years ago

I think @paperrobots is unfortunate, but does have a valid point as it's called wordmove, not rsync-plugin, so you'd assume (without any command line knowledge), it didn't touch the other files.

However, I wonder if a better workflow would be to pull a live site to your local and in turn pull everything down that is on the production server. You can then suitably ignore the files (or never change them), so they don't go anywhere

Or am I missing something?

alessandro-fazzi commented 8 years ago

@simonmacfadyen when you say

However, I wonder if a better workflow would be to pull a live site to your local and in turn pull everything down that is on the production server. You can then suitably ignore the files (or never change them), so they don't go anywhere

it's exactly what I'd do to work on a polluted host. The @paperrobots 's point is to be aware of the behavior, I think.

ghost commented 8 years ago

Indeed. :+1:

Not sure you can protect people fully from themselves, though.

kevinwhoffman commented 8 years ago

I'm not sure if this is possible, but it feels like a situation where the --simulate option could be improved to give a better idea of what files will be modified during the migration. In its current form, the --simulate option does not give you an accurate preview of the specific files that will be affected. Doing so would go a long way in preventing this kind of situation.

I'm fully aware that rsync does not distinguish between WP/non-WP files, but I still catch myself deleting files that I didn't intend to delete because I copied a Movefile from another project that doesn't have the exact same remote setup.

alessandro-fazzi commented 8 years ago

@kevinwhoffman you should try to pass --dry-run to rsync through the Movefile in order to achieve that result. For example:

ssh:
    host: "mt3"
    rsync_options: "--verbose --itemize-changes --dry-run"
~/dev/prj ·  git:(master) 1M
⟩ wordmove push -t                                                                                                                                                    [ruby 2.2.3]

▬▬ ✓ Using Movefile: ./Movefile ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

▬▬ ✓ Pushing Themes ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
   remote | put_directory: /Users/fuzzy/dev/prj/wp-content/themes /nfs/c04/h07/mnt/163339/users/.home/domains/prj.demo.welaika.com/html/wp-content/themes .git/ .gitignore .sass-cache/ node_modules/ bin/ tmp/* Gemfile* Movefile wp-config.php wp-content/*.sql.gz
I, [2016-05-30T12:23:30.338565 #12591]  INFO -- : rsync --progress -e 'ssh mt3' -rlpt --compress --omit-dir-times --delete --verbose --itemize-changes --dry-run --exclude .git/ --exclude .gitignore --exclude .sass-cache/ --exclude node_modules/ --exclude bin/ --exclude tmp/\* --exclude Gemfile\* --exclude Movefile --exclude wp-config.php --exclude wp-content/\*.sql.gz /Users/fuzzy/dev/prj/wp-content/themes/ :/nfs/c04/h07/mnt/163339/users/.home/domains/prj.demo.welaika.com/html/wp-content/themes
sending incremental file list
*deleting   prj/assets/javascripts/application.js
*deleting   prj/assets/stylesheets/screen.css
*deleting   prj/assets/stylesheets/bemo.css

sent 3,901 bytes  received 313 bytes  1,204.00 bytes/sec
total size is 1,321,015  speedup is 313.48 (DRY RUN)

For sure a bunch of advanced features are foreseeable along this path, but this is a no-brainer and functional way to test your pushes.

veganista commented 8 years ago

Just so it's clear, let's say that i have a sub directory in my wordpress install called /shop will adding /shop to the exclude file leave the directory untouched?

alessandro-fazzi commented 8 years ago

Yes. But please, test with --dry-run before doing a real sync. Sometimes rsync's exclusion list is not so trivial to understand (heading slash, trailing slash, dot folders, etc. could be someway confusing)

veganista commented 8 years ago

rsync_options: "--verbose --itemize-changes --dry-run" works perfectly.

Could it be worth adding this automatically to the --simulate option when using rsync?

alessandro-fazzi commented 8 years ago

It works and you are not seeing any *deleting in the log, right? 😜

It could be a nice feature suggestion :) ready to send me a PR? 🙌

Or at least it could be useful to open an issue with the suggestion, so that we can tag it accordingly and keep track of it until the next dev session.

kevinwhoffman commented 8 years ago

+1 @veganista's suggestion to incorporate --dry-run into --simulate since that was the output I originally expected when using --simulate anyways.

@pioneerskies Sorry I missed when you originally posted https://github.com/welaika/wordmove/issues/271#issuecomment-222463465 but that is exactly what I'm looking for!

veganista commented 8 years ago

@pioneerskies Yeah, i did a bit of testing with the options enabled, without, with the folder excluded and without. Got the output expected each time.

I'll do a little bit more testing tomorrow and see what i can do about a PR

alessandro-fazzi commented 6 years ago

217

alessandro-fazzi commented 6 years ago

Added documentation about mirroring behavior in main README.

Updated --simulate behavior in 13af387 and in 2aa3e8f.

They'll be in the next release.

I'll close this one, but feel free to comment.