Permissions issues with staging and production (and sometimes local)

[mcheck]

I can't determine if there is a permissions issue or possibly a url rewrite issue?

Many plugins (for instance) fail to allow deletion from the web ui. At first I thought permissions issue so

cap staging genesis:permissions

On staging for instance

vagrant@staging:/var/www/mysite.com/master/current/web/wp-content/plugins
drwxrwxr-x 4 deploy www-data 4096 Oct 17 19:07 .
drwxrwxr-x 5 deploy www-data 4096 Oct 17 20:01 ..
drwxrwxrwx 2 deploy www-data 4096 Oct 17 19:04 hello-dolly
-rwxrwxr-x 1 deploy www-data   30 Oct 16 19:12 index.php

Trying to delete the hello dolly plugin from the wp ui, presents the Connection Information, implying the permissions are not allowed.

For kicks,

sudo chmod -R 0777 /var/www/mysite.com/master/current/web

then

vagrant@staging:/var/www/mysite.com/master/current/web/wp-content/plugins
drwxrwxrwx 4 deploy www-data 4096 Oct 17 19:07 .
drwxrwxrwx 5 deploy www-data 4096 Oct 17 20:01 ..
drwxrwxrwx 2 deploy www-data 4096 Oct 17 19:04 hello-dolly
-rwxrwxrwx 1 deploy www-data   30 Oct 16 19:12 index.php

I can delete hello dolly from the wp ui on 'local', but when I try to add plugins from the either the online plugin repo or even uploading, I get a blank screen and no errors on many (most) of the plugins.

I thought maybe url rewrites as well, because while some plugins work when installing from the wp ui, others do not. I cannot determine the criteria.

I have turned on all php errors in wp-config.php with

@ini_set( 'log_errors', 'Off' );
@ini_set( 'display_errors', 'On' );
@ini_set( 'error_reporting', E_ALL );
define( 'WP_DEBUG', true );
define( 'WP_DEBUG_LOG', false );
define( 'WP_DEBUG_DISPLAY', true );

When trying to install wp-client-lite on 'local' gives a blank screen (even with all php error debugging on) and no errors reported in apache logs and the url is

http://local.mysite.com/wp-admin/update.php?action=install-plugin&plugin=wp-client-client-portals-file-upload-invoices-billing&_wpnonce=d1a0dc7dd0

Trying to instal WP Widget Preview on 'local' works fine with url

http://local.mysite.com/wp-admin/update.php?action=install-plugin&plugin=wp-widget-preview&_wpnonce=4b70a21e76

Maybe a missing php extension?

All this odd too because, of course, adding and deleting media works without issue (so far).

Thanks.

Mike

[mcheck]

An update: on staging and production, I can delete plugins after I

sudo chown -R www-data:www-data web/

However, it still will not allow me to add them. Perhaps a /tmp directory or other needs changing to be owned by the www-data user?

Mike

[ericclemmons]

I'll have to test this. We add all of our plugins to the filesystem, not through the admin, so that git is used to manage all code, and the admin only manages the content.

[EvanK]

@mcheck Can you test whether this is still happening as of v0.2.21?

[mcheck]

Yes, still happening. wp-content after deploy:

 drwxr-xr-x  5 deploy www-data 4096 Oct 22 15:55 .
 drwxr-xr-x  5 deploy www-data 4096 Oct 17 21:45 ..
 -rw-rw-r--  1 deploy www-data   28 Oct 22 14:00 index.php
 drwxr-xr-x 12 deploy www-data 4096 Oct 22 15:55 plugins
 drwxr-xr-x  8 deploy www-data 4096 Oct 22 13:54 themes
 drwxr-xr-x  2 deploy www-data 4096 Oct 22 15:55 upgrade

Cannot delete plugins from the gui.

If I give the web server group full privileges:

sudo chmod -R 775 web

drwxrwxr-x  5 deploy www-data 4096 Oct 22 16:01 .
drwxrwxr-x  5 deploy www-data 4096 Oct 17 21:45 ..
-rw-rw-r--  1 deploy www-data   28 Oct 22 14:00 index.php
drwxrwxr-x 12 deploy www-data 4096 Oct 22 16:01 plugins
drwxrwxr-x  8 deploy www-data 4096 Oct 22 13:54 themes
drwxrwxr-x  2 deploy www-data 4096 Oct 22 16:01 upgrade

Still cannot delete or add plugins.

Only when I change the owner to the web server

sudo chown -R www-data:www-data web

drwxrwxr-x  5 www-data www-data 4096 Oct 22 16:06 .
drwxrwxr-x  5 www-data www-data 4096 Oct 17 21:45 ..
-rw-rw-r--  1 www-data www-data   28 Oct 22 14:00 index.php
drwxrwxr-x 12 www-data www-data 4096 Oct 22 16:01 plugins
drwxrwxr-x  8 www-data www-data 4096 Oct 22 13:54 themes
drwxrwxr-x  2 www-data www-data 4096 Oct 22 16:01 upgrade

Themes and plugins are installable from the gui. I say most plugins because trying some random plugins like above (wp-client-lite) results in blank screen even with www-data owning the entire web folder.

Perhaps something in apache root folder permissions config that disallow group owner to modify files?

[EvanK]

I do know that permissions will never be quite right on local, since the codebase is mounted via nfs under /vagrant/web, and ends up with an owner of dialout.

That said...since the deploy user is a member of the www-data group, it should be safe to have everything owned by user and group www-data, as long as we set permissions of 775 for directories and 664 for files?

[mcheck]

It should be ok in theory, but it doesn't work as expected. Deleting or adding plugins from the GUI bring up the FTP connection dialog because the permissions are not correct.

Mike

[ericclemmons]

We'll have to look into this more. Truth be told, this isn't part of the workflow that this tool was built around. staging & production are by-products of the deployment, and changes to plugins & whatnot are done on local & deployed out. (Otherwise, plugins are out of sync with Github!)

We still manage plugins on local with no problem:

Also, we've found the best way to remove a plugin across environments is actually removing the plugin from wp-content/plugins, as WordPress will gracefully remove it for you!

[mcheck]

OK, I see that it is not the intended workflow. It does make sense to deploy all from a repo.

This topic gets into my other issue on workflow: there is transient data in that workflow that I would like considered for integration. For instance, when adding a plugin to local, commit and deploy to staging, there are certain defaults or settings that I would like to push to staging, then perhaps to production, and that is what I would use the cap genesis:up/down:db for. The plugin settings (and sometimes themes and other configuration items) are stored in the db and I would like a way to move them around to speed testing.

More detail in my workflow: add a plugin to local, adjust the configuration (which is saved to db or files), then push the whole thing to staging to be tested. During testing, I might change some more configuration, and then want to pull in those settings (maybe there are a lot of configuration items and I don't want to go through every one of them, e.g. s2member). That's where I (would) use the cap genesis:up/down. I don't have to push/pull the whole thing at once, I could use cap tasks to selectively move the database, config files, etc.

How else do we manage the configurations/settings in the current workflow? Do we set all the configs on local, then again on staging, then production?

I can use the genesis:up/down, I just need to adjust for the root url setting (and perhaps others that are specific to the environment?)

Again, maybe there is something I don't understand on the configs and settings of plugins and themes, how they are tested and set in the various envs.

And I haven't said yet: this framework is great. Thanks for putting it together. I'm happy to help if I can.

Mike

[mcheck]

OK, found the reason deploy:www-data does not allow plugin management on sites that are NOT local.

It turns out that WP does a test when adding/deleting plugins by writing a file and then testing it to see if the owner of that file is the same owner it is running as. So no matter the permissions, deploy:www-data will fail.

The test is born of the WP upgrade process and was extended to plugins management - I'm not sure why the same error does not appear with themes. The following code is from the get_filesystem_method() method in the wp-admin/includes/file.php file:

 if( function_exists('getmyuid') && function_exists('fileowner') ){
     $temp_file = wp_tempnam();
     if ( getmyuid() == fileowner($temp_file) )
         $method = 'direct';
     unlink($temp_file);
 }

So changing the filesystem management method to 'direct' does not perform the test as described here

I added the following to the wp-config.php file for staging and production:

 define ( 'FS_METHOD', 'direct' );

And plugin management works as expected.

Further, I found a permissions guide for adding and removing files and directories by the web server.

Based on this, I propose to change the permissions task in server.rb to:

task :permissions do
    run "sudo find #{remote_web} -follow -type d -exec chmod 775 {} \\; -exec chmod g+s {} \\;"
    run "sudo find #{remote_web} -follow -type f -exec chmod 664 {} \\;"

I'm open for comments, of course.

Thanks,

Mike

[AntJanus]

@mcheck Wouldn't the plugin installation disabling be a necessary and wanted feature? All changes should take place locally, including plugin management (adding/deleting plugins). Thus, when you install a plugin locally, you're able to save your changes, add them to your git repository, and deploy your application to staging and later to production.

The enabling/disabling of the plugin, then, would fall on staging/production, same as addition and revision of content and other settings, as well as comments. Unfortunately, SQL doesn't work in the same manner as git (wouldn't that be wonderful?) so you can't really compare changes and push/pull like you would with files. Blowing away staging/production database would also be counter productive since non-overwriteable changes should cascade from production down and you can never push up (from staging to production) in a full-rewrite mode.

For example, production gets comment and content changes. That should be pulled into staging and overwrite staging (for new content addition). The local machine should always get the latest copy from staging as well.

Wouldn't this flow make more sense? I'm just trying to think of cases where you'd want to install (download + enable) a plugin on production or staging environment before doing so locally.

[mcheck]

@AntJanus Yes, I think the plugin/theme/updating should be disabled in production. Not necessarily in staging where we are changing things around to see what works and what doesn't. That includes plugins and their settings.

I suggest we test to see if production would benefit from define( 'DISALLOW_FILE_MODS', true ); reference

I agree, once the site is in full swing, it not likely we are ever replacing a production db with local or staging. But on initial deploy we might. So the first deploy with all new plugins, settings, content and etc. would be a complete db push to production. From then on we would pull the db from production, make changes locally, push the deploy and db to staging, rinse, repeat, until pushing just the code to production. Any configs have to be manual e.g. activating plugins, setting their defaults, etc.

Wouldn't this flow make more sense? I'm just trying to think of cases where you'd want to install (download + enable) a plugin on production or staging environment before doing so locally.

For production, there is no case where this makes sense, I say disable it. But for staging, I would use it test on servers that are not my development machine. Since staging is somewhere in the cloud, it duplicates production, and it allows clients (or project members) to work on it for testing purposes - this might include changing configs as well as content. I would pull db/files from production to local, make some changes, deploy and push db/files to staging, mess around with it, pull db/files to local make more changes, etc. until deploying to production and manually make config changes.

But adding the FS_METHOD and changing the cap task as above only gives staging and production something that should have had to being with. I have the flexibility to add / remove it from my workflow, but either way it should work as expected.

What would be REALLY great is to isolate the db changes made during each update and apply them, with their seed data (from code) to staging then to production. This is more like the workflow we have with rails apps. When we change something locally, we add seed data as code or as a cap task for that function and apply it to the staging db on deploy. But that is fundamentally diff than the way WP handles plugins and their data for instance. But it would be neat. Really neat.

[AntJanus]

@mcheck Hmm.. okay, I get the case for staging. A Vagrant machine should be able to replace the need for staging testing but Vagrant does not always = true production environment.

I still think that a plugin should be installed locally first and then pushed up to staging. I don't believe that any code changes (and this would classify as one) should be done in a staging environment.

In my head, I'd follow the following workflow (with plugins):

1. install locally and test locally. Does it work?
2. if yes, commit the code (and the plugin), push to repo
3. deploy branch with plugin to staging
4. enable plugin at staging
5. test. Does it work? If not, go back to step 1.
6. if yes, deploy code to production.
7. enable plugin in production.

I would really love seeding in Wordpress as well but I don't believe there is a workable solution just yet.

As far as uploading the db initially, that's exactly what this tool can do. I just meant that you can't manipulate databases in a git-like fashion. :/ Though, I think that would be awesome if there was a good solution for that (a built in one would be awesome too).

[mcheck]

@AntJanus Agree with your workflow. Best put into list like that. Thanks for considering all this. For the changes to wp_config, and deploy files, do you recc a pull request? Or just leave here for later inclusion?

And in the meantime re:local changes to bower_components (for the wp-config and deploy files), what is a best practice to have them used across all my development machines?

• remove them from .gitignore ?
• or is there a custom lib or area to override bower_component functions and libs?

[EvanK]

I believe the permissions issues here should be resolved as of v0.2.32.

@mcheck Per your last comment (sorry for delayed response, it got lost in the fray of my inbox), my SOP for testing custom changes is to copy my local genesis repo into bower_components/genesis-wordpress and edit it directly while I'm testing, then commit + push to a feature branch and make a PR once it's working.