Unable to sync local project files

[alexciarlillo]

I am trying to sync my compiled assets for my sage-based theme with no luck.

The first issue was that my theme folders are not named "sage", but are based on the project I am working on. In this case the target theme is "wordpress-base-theme" which is my fork of Sage that I base further themes off of. So I found the appropriate lines in the deploy/main.yml to setup syncing local project files:

project_local_files:
  - name: compiled assets
    src: "{{ project.local_path }}/web/app/themes/{{ project.theme }}/dist"
    dest: "web/app/themes/{{ project.theme }}"

I then added a theme variable to my staging/wordpress_sites[site]

wordpress_sites:
  main-site:
    site_hosts:
      - staging.mysite.com
    local_path: '../main-site.dev' # path targeting local Bedrock site directory (relative to Ansible root)\
    theme: wordpress-base-theme
 ...

And then did the deploy:

$> ./deploy.sh staging main-site 
[-- cut a bunch of output --]

 _____________________________________________________________
< TASK: deploy | Run pre_build_commands_local on Ansible host >
 -------------------------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

skipping: [staging]
 _________________________________________
< TASK: deploy | Copy project local files >
 -----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

changed: [staging -> 127.0.0.1] => (item={'dest': u'web/app/themes/wordpress-base-theme', 'src': u'../main-site.dev/web/app/themes/wordpress-base-theme/dist', 'name': 'compiled assets'})
 _______________________________________
< TASK: deploy | Copy project templates >
 ---------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

changed: [staging] => (item={'dest': '.env', 'src': 'roles/deploy/templates/env.j2', 'name': '.env config'})

[-- cut the rest of output --]
staging                    : ok=16   changed=10   unreachable=0    failed=0   

I read up a but on the synchronize documentation from ansible, but everything looks fine. I tried explicitly defining the "archive" and/or "dirs" options with no luck. The only odd thing I noticed was that the info message for that task has [staging -> 127.0.0.1] where every other has [staging] but my guess from limited ansible usage is that is specifying that is execute the rsync command on the localhost.

I feel like I must be missing something obvious, any help would be appreciated.

Lastly, here is the full debug output of the "Copy project local files" task in case it provides any insights:

 _________________________________________
< TASK: deploy | Copy project local files >
 -----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

<127.0.0.1> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1430245912.06-216939515803640 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1430245912.06-21693951580364
0 && echo $HOME/.ansible/tmp/ansible-tmp-1430245912.06-216939515803640']
<127.0.0.1> PUT /tmp/tmpPCMfDJ TO /home/ciarlill/.ansible/tmp/ansible-tmp-1430245912.06-216939515803640/synchronize
<127.0.0.1> EXEC ['/bin/sh', '-c', u'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /home/ciarlill/.ansible/tmp/ansible-tmp-1430245912.06-216939515803640/synchronize; rm -r
f /home/ciarlill/.ansible/tmp/ansible-tmp-1430245912.06-216939515803640/ >/dev/null 2>&1']

changed: [staging -> 127.0.0.1] => (item={'dest': u'web/app/themes/wordpress-base-theme', 'src': u'../main-site.dev/web/app/themes/wordpress-base-theme/dist', 'name': 'compiled as
sets'}) => {"changed": true, "cmd": "rsync --delay-updates -F --compress --archive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/home/ciarl
ill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme/dist\" \"ciarlill@staging:/home/ciarlill/main-site/releases/20150428143149/web/app/themes/wo
rdpress-base-theme\"", "item": {"dest": "web/app/themes/wordpress-base-theme", "name": "compiled assets", "src": "../main-site.dev/web/app/themes/wordpress-base-theme/dist"}, "msg
": "cd+++++++++ dist/\ncd+++++++++ dist/fonts/\n<f+++++++++ dist/fonts/FontAwesome.otf\n<f+++++++++ dist/fonts/fontawesome-webfont.eot\n<f+++++++++ dist/fonts/fontawesome-webfont.
svg\n<f+++++++++ dist/fonts/fontawesome-webfont.ttf\n<f+++++++++ dist/fonts/fontawesome-webfont.woff\n<f+++++++++ dist/fonts/fontawesome-webfont.woff2\n<f+++++++++ dist/fonts/glyp
hicons-halflings-regular.eot\n<f+++++++++ dist/fonts/glyphicons-halflings-regular.svg\n<f+++++++++ dist/fonts/glyphicons-halflings-regular.ttf\n<f+++++++++ dist/fonts/glyphicons-h
alflings-regular.woff\n<f+++++++++ dist/fonts/glyphicons-halflings-regular.woff2\ncd+++++++++ dist/images/\n<f+++++++++ dist/images/ginkgo.png\n<f+++++++++ dist/images/give_now.pn
g\n<f+++++++++ dist/images/make_gift.png\n<f+++++++++ dist/images/ou_front_logo_narrow.png\ncd+++++++++ dist/images/header/\n<f+++++++++ dist/images/header/banner1.jpg\n<f++++++++
+ dist/images/header/banner10.jpg\n<f+++++++++ dist/images/header/banner2.jpg\n<f+++++++++ dist/images/header/banner3.jpg\n<f+++++++++ dist/images/header/banner4.jpg\n<f+++++++++ 
dist/images/header/banner5.jpg\n<f+++++++++ dist/images/header/banner6.jpg\n<f+++++++++ dist/images/header/banner7.jpg\n<f+++++++++ dist/images/header/banner8.jpg\n<f+++++++++ dis
t/images/header/banner9.jpg\ncd+++++++++ dist/images/header/thumbs/\n<f+++++++++ dist/images/header/thumbs/banner1.jpg\n<f+++++++++ dist/images/header/thumbs/banner10.jpg\n<f+++++
++++ dist/images/header/thumbs/banner2.jpg\n<f+++++++++ dist/images/header/thumbs/banner3.jpg\n<f+++++++++ dist/images/header/thumbs/banner4.jpg\n<f+++++++++ dist/images/header/th
umbs/banner5.jpg\n<f+++++++++ dist/images/header/thumbs/banner6.jpg\n<f+++++++++ dist/images/header/thumbs/banner7.jpg\n<f+++++++++ dist/images/header/thumbs/banner8.jpg\n<f++++++
+++ dist/images/header/thumbs/banner9.jpg\ncd+++++++++ dist/scripts/\n<f+++++++++ dist/scripts/jquery.js\n<f+++++++++ dist/scripts/jquery.js.map\n<f+++++++++ dist/scripts/main.js\
n<f+++++++++ dist/scripts/main.js.map\n<f+++++++++ dist/scripts/modernizr.js\n<f+++++++++ dist/scripts/modernizr.js.map\ncd+++++++++ dist/styles/\n<f+++++++++ dist/styles/editor-s
tyle.css\n<f+++++++++ dist/styles/editor-style.css.map\n<f+++++++++ dist/styles/main.css\n<f+++++++++ dist/styles/main.css.map\n", "rc": 0, "stdout_lines": ["cd+++++++++ dist/", "
cd+++++++++ dist/fonts/", "<f+++++++++ dist/fonts/FontAwesome.otf", "<f+++++++++ dist/fonts/fontawesome-webfont.eot", "<f+++++++++ dist/fonts/fontawesome-webfont.svg", "<f++++++++
+ dist/fonts/fontawesome-webfont.ttf", "<f+++++++++ dist/fonts/fontawesome-webfont.woff", "<f+++++++++ dist/fonts/fontawesome-webfont.woff2", "<f+++++++++ dist/fonts/glyphicons-ha
lflings-regular.eot", "<f+++++++++ dist/fonts/glyphicons-halflings-regular.svg", "<f+++++++++ dist/fonts/glyphicons-halflings-regular.ttf", "<f+++++++++ dist/fonts/glyphicons-half
lings-regular.woff", "<f+++++++++ dist/fonts/glyphicons-halflings-regular.woff2", "cd+++++++++ dist/images/", "<f+++++++++ dist/images/ginkgo.png", "<f+++++++++ dist/images/give_n
ow.png", "<f+++++++++ dist/images/make_gift.png", "<f+++++++++ dist/images/ou_front_logo_narrow.png", "cd+++++++++ dist/images/header/", "<f+++++++++ dist/images/header/banner1.jp
g", "<f+++++++++ dist/images/header/banner10.jpg", "<f+++++++++ dist/images/header/banner2.jpg", "<f+++++++++ dist/images/header/banner3.jpg", "<f+++++++++ dist/images/header/bann
er4.jpg", "<f+++++++++ dist/images/header/banner5.jpg", "<f+++++++++ dist/images/header/banner6.jpg", "<f+++++++++ dist/images/header/banner7.jpg", "<f+++++++++ dist/images/header
/banner8.jpg", "<f+++++++++ dist/images/header/banner9.jpg", "cd+++++++++ dist/images/header/thumbs/", "<f+++++++++ dist/images/header/thumbs/banner1.jpg", "<f+++++++++ dist/image
s/header/thumbs/banner10.jpg", "<f+++++++++ dist/images/header/thumbs/banner2.jpg", "<f+++++++++ dist/images/header/thumbs/banner3.jpg", "<f+++++++++ dist/images/header/thumbs/ban
ner4.jpg", "<f+++++++++ dist/images/header/thumbs/banner5.jpg", "<f+++++++++ dist/images/header/thumbs/banner6.jpg", "<f+++++++++ dist/images/header/thumbs/banner7.jpg", "<f++++++
+++ dist/images/header/thumbs/banner8.jpg", "<f+++++++++ dist/images/header/thumbs/banner9.jpg", "cd+++++++++ dist/scripts/", "<f+++++++++ dist/scripts/jquery.js", "<f+++++++++ di
st/scripts/jquery.js.map", "<f+++++++++ dist/scripts/main.js", "<f+++++++++ dist/scripts/main.js.map", "<f+++++++++ dist/scripts/modernizr.js", "<f+++++++++ dist/scripts/modernizr
.js.map", "cd+++++++++ dist/styles/", "<f+++++++++ dist/styles/editor-style.css", "<f+++++++++ dist/styles/editor-style.css.map", "<f+++++++++ dist/styles/main.css", "<f+++++++++ 
dist/styles/main.css.map"]}

[swalkinshaw]

What exactly is the issue? The uploaded files don't appear on the remote server?

Just asking since it completes without errors.

In your verbose output you can see the exact command it's running:

rsync --delay-updates -F --compress --archive --rsh 'ssh -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/home/ciarl ill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme/dist\" \"ciarlill@staging:/home/ciarlill/main-site/releases/20150428143149/web/app/themes/wo rdpress-base-theme\

At first glance this looks correct to me (assuming those paths are correct).

[swalkinshaw]

changed: [staging -> 127.0.0.1] is normal since it's running a local command as you guessed.

[fullyint]

looks like the destination is resolving to a path in the home directory for some reason ciarlill@staging:/home/ciarlill/main-site/releases/20150428143149/web/app/themes/wo rdpress-base-theme\ but probably ought to resolve to a path in something like /srv/www/main-site/etc.

[alexciarlillo]

Sorry I should have clarified that the uploaded files are not appearing. And also should have noted that it is going to the home directory because I am just testing out the deployment process before I blow up my existing staging site and re-deploy with ansible.

[swalkinshaw]

You can try running that command manually to see what the output is:

rsync --delay-updates -F --compress --archive --rsh 'ssh -S none -o StrictHostKeyChecking=no'  \"/home/ciarlill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme/dist\" \"ciarlill@staging:/home/ciarlill/main-site/releases/20150428143149/web/app/themes/wordpress-base-theme\"

[alexciarlillo]

Yes I just did try that and it does work, but only if I remove the escaped double quotes on everything...

Otherwise I get:

$> rsync --delay-updates -F --compress --recursive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/home/ciarlill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme/dist\" \"ciarlill@staging:/home/ciarlill/main-site/releases/20150428150024/web/app/themes/wordpress-base-theme\"

"ciarlill@lib-web-lp001.mgt.private's password: 

lib-web-lp001.mgt.private being the internal network name of the server I am targeting.

So if I remove the escaped quotes from the last argument:

$> rsync --delay-updates -F --compress --recursive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' \"/home/ciarlill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme/dist\" ciarlill@staging:/home/ciarlill/main-site/releases/20150428150024/web/app/themes/wordpress-base-theme   
rsync: change_dir "/home/ciarlill/Development/environments/bedrock/bedrock-ansible//"/home/ciarlill/Development/environments/bedrock/main-site.dev/web/app/themes/wordpress-base-theme" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.0]

$> pwd
/home/ciarlill/Development/environments/bedrock/bedrock-ansible

And lastly, removing all escaped double quotes:

$> rsync --delay-updates -F --compress --recursive --rsh 'ssh  -S none -o StrictHostKeyChecking=no' --out-format='<<CHANGED>>%i %n%L' /home/ciarlill/Development/environments/bedr
ock/main-site.dev/web/app/themes/wordpress-base-theme/dist ciarlill@staging:/home/ciarlill/main-site/releases/20150428150024/web/app/themes/wordpress-base-theme                   
<<CHANGED>>cd+++++++++ dist/
<<CHANGED>>cd+++++++++ dist/fonts/
... [cut rest of output] ....

Works like a charm... seems like this may be an ansible thing, not bedrock-ansible. Very strange all around.

[swalkinshaw]

I actually think it's working (at least partly). Ansible probably just escapes those strings for internal usage.

But if you notice in your last working example of the manual command, you see output like:

<<CHANGED>>cd+++++++++ dist/
<<CHANGED>>cd+++++++++ dist/fonts/

And if you look at your original Ansible logs, you see the same type of output for each individual file.

It definitely seems like rsync is uploading the files. Maybe something is happening after that?

[alexciarlillo]

Good point. I think I know what the problem is... I declare my themes as a composer dependency from a git repo with the 'dist' directory gitignored, so running 'composer install' is probably destroying the entire directory and the synchronized files along with it... doh.

I know somewhere in the docs you guys DO NOT recommend making themes a dependency of bedrock projects. Now I understand why. I will close this once I verify this is the case.

[alexciarlillo]

Yes that was the problem. Resolved for my use case by moving "Copy project local file" to run after the "Run post_build_commands" task. Apologies for raising an issue related to something you specifically suggest NOT to do in the documentation. :grimacing:

[kalenjohnson]

I've had this issue as well when installing the theme as a Composer dependency and not part of the Bedrock project. I wonder if "Copy project local file" could be placed underneath "Run post_build_commands" for this reason.

[mAAdhaTTah]

@kalenjohnson @alexciarlillo That's good to know; I have my themes set up as composer dependencies currently but haven't moved everything over to ansible deployments. Will have to remove that. So are you guys copying up the entire theme folder with ansible or how are you handling it?

(Assuming you're not committing your theme files to your site bedrock repo?)

[alexciarlillo]

@mAAdhaTTah I just modified the ansible deploy task to copy up the assets after running 'composer install'. Because of the way our team is setup it makes most sense to have themes as their own repo. The others on my team mostly do styling specific work or change a bit of PHP here and there, but are not "ops" (or even full time devs) by any means. Since I am the only person who touches that stuff I like to keep them separate.

I think on my personal site (when I get to updating it) or small freelance stuff I would be fine setting up the project and building the theme directly in the bedrock repo and think of it more as a unified "app".