Question / Issue (Errcode: 13 "Permission denied") Can't pull database, error during the backup step

MarkErik commented 5 years ago

Hello!

I have a WP instance running in docker containers (docker-compose.yml and movefile.yml below).

I am unable to complete a pull from the remote instance, as it throws and error when it gets to the step of backing up the database: _/wpfiles/wp-content/dump.sql' (Errcode: 13 "Permission denied")

From ls -la: drwxr-xr-x 7 www-data www-data 4096 Sep 7 16:25 wp-content

From a previous post, it appears that only root has write permission to that folder.

What are some secure ways to address this?

(I was thinking that I could modify the wordmove code to try to save the db dump my user's home folder, but my preference is not to modify the source code...)

Thank you for your help!

--- docker-compose ---

version: '3.5'

services:

  db:
    image: mariadb
    container_name: db
    restart: unless-stopped
    env_file: .env
    environment:
     - MYSQL_DATABASE=wordpress
    volumes:
     - ./www/wp_DB:/var/lib/mysql
    ports:
     - "127.0.0.1:3306:3306"
       command: ['--default-authentication-plugin=mysql_native_password','--character-set-server=utf8mb4','--collation-server=utf8mb4_unicode_ci']

  wordpress:
      depends_on: 
        - db
      image: wordpress:fpm
      container_name: wordpress
      restart: unless-stopped
      env_file: .env
      environment:
        - WORDPRESS_DB_HOST=db:3306
        - WORDPRESS_DB_USER=$MYSQL_USER
        - WORDPRESS_DB_PASSWORD=$MYSQL_PASSWORD
        - WORDPRESS_DB_NAME=wordpress
      volumes:
        - ./www/wp_files:/var/www/html

  webserver:
      depends_on:
        - wordpress
      image: nginx:alpine
      container_name: webserver
      restart: unless-stopped
      ports:
        - "80:80"
        - "443:443"
      volumes:
        - ./www/wp_files:/var/www/html:ro
        - ./configs/nginx/conf.d:/etc/nginx/conf.d:ro
        - ./configs/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
        - ./configs/certbot:/etc/letsencrypt:ro
        - ./logs/nginx:/var/log/nginx

#(Note: I only include the word move container on my local docker-compose) 
wordmove:
    tty: true
    build:
      context: ./docker-images/wordmove/
    container_name: wordmove  
    restart: unless-stopped
    volumes:
      - ./www/wp_files/:/www
      # Movefile
      - ./configs/wordmove/:/home/
      # For ssh
      - ${HOME}/.ssh/:/root/.ssh:ro
      - /etc/passwd:/etc/passwd:ro

--- Dockerfile for wordmove: ---

FROM welaika/wordmove:alpine

RUN wp cli update --yes

--- Movefile ---

global:
  sql_adapter: default

local:
  vhost: http://localhost
  wordpress_path: /www/ 

  database:
    name: "wordpress"
    user: "root"
    password: "<%= ENV['LOCAL_DB_PASS'] %>" 
    host: "db" 
    port: "3306"
    charset: "utf8mb4"

staging-server:
  vhost: https://xyz.xyz.com
  wordpress_path: /home/xyz/staging-site/www/wp_files/

  database:
    name: wordpress
    user: "<%= ENV['STAGING_DB_USER'] %>"
    password: "<%= ENV['STAGING_DB_PASS'] %>"
    host: "127.0.0.1"
    port: 3306 
    charset: "utf8mb4"

  exclude:
    - '.git/'
    - '.gitignore'
    - '.sass-cache/'
    - 'node_modules/'
    - 'bin/'
    - 'tmp/*'
    - 'Gemfile*'
    - 'Movefile'
    - 'movefile'
    - 'movefile.yml'
    - 'movefile.yaml'
    - 'wp-config.php'
    - 'wp-content/*.sql.gz'
    - '*.orig'

  ssh:
    host: staging-server

alessandro-fazzi commented 4 years ago

Isn't it related to the fact you are declaring the volume as read only in your webserver config?

- ./www/wp_files:/var/www/html:ro

MarkErik commented 4 years ago

Hi @pioneerskies thank you for your response.

In the docker-compose file, my understanding is that the :ro read-only flag limits the Nginx docker service from writing to the shared directory on the host, whereas the SQL commands would be executed via the SSH connection, thus not affected by the individual constraints on the docker services.

In this case the docker service that has write access to the host directory is the Wordpress service, that's why the previous Wordmove commands execute OK.

alessandro-fazzi commented 4 years ago

Let me better understand:

are you doing a pull such as wordmove pull -d -e staging-server?
how is staging-server translated into you ssh_config?
in the error /wp_files/wp-content/dump.sql' (Errcode: 13 "Permission denied") have you altered the path? I can't figure where the path /wp_files/wp-content/dump.sql should be since I can't read it in any configuration you provided
from where the command

From ls -la:
drwxr-xr-x 7 www-data www-data 4096 Sep 7 16:25 wp-content

is launched? I mean: from which host and from which pwd?

MarkErik commented 4 years ago

@pioneerskies Thank you for the detailed questions, I will try to provide more info below, and I hope this will provide more clarity as to why I am hoping to find a solution that will allow the mysqldump command to complete seeing as how it currently doesn't have access to write to the wp-content folder which is on the remote server located at ~/wp-web/www/wp_files/wp-content

wp-web is my project name, and www/wp_files is the directory that is linked as a volume for the wordpress container.

Background info:

"Staging" server running Wordpress (in Docker) is hosted on a DigitalOcean virtual server
I have root SSH access turned off, and instead SSH into the remote server using the user 'staging-user' (name changed here for privacy. staging-user also has sudo privilege.
MacOS is my local machine, and have made a 'config' entry for the remote server which here I will call "do-xyz" in the ~/.ssh/config file on my local host machine and have the access keys in the ~/.ssh/ directory. I can run 'ssh do-xyz' from my local machine to access the remote server
When I am working inside the Wordmove docker instance on my local machine (via docker exec -it wordmove bash), I can also ssh into the remove server from the command line ('ssh do-xyz'). This is because in the docker-config the .ssh folder is linked as read-only into the Wordmove container:

wordmove:
    volumes:
      ./www/wp_files/:/www
      # Movefile
      - ./configs/wordmove/:/home/
      # For ssh
      - ${HOME}/.ssh/:/root/.ssh:ro
      - /etc/passwd:/etc/passwd:ro

Each time I enter the Wordmove docker instance I run the following 2 commands: eval ssh-agent -s & ssh-add /root/.ssh/do-xyz-key
Running wordmove doctor inside the Wordmove docker instance doesn't report any errors

To answer the questions:

are you doing a pull such as wordmove pull -d -e staging-server?

Yes, that is the command that I execute from within the Wordmove docker instance. Here is the output with private elements obscured

▬▬ Pulling Database ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
    local | mysqldump --host=db --port=3306 --user=root --password=local-password --result-file="/www/wp-content/local-backup-1574065594.sql" wordpress
    local | gzip -9 -f "/www/wp-content/local-backup-1574065594.sql"
   remote | mysqldump --host=127.0.0.1 --port=3307 --user=xxx-xxx --password=xxx-xxx --result-file="/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql" wordpress
Traceback (most recent call last):
    15: from /usr/local/bundle/bin/wordmove:23:in `<main>'
    14: from /usr/local/bundle/bin/wordmove:23:in `load'
    13: from /usr/local/bundle/gems/wordmove-4.0.1/exe/wordmove:6:in `<top (required)>'
    12: from /usr/local/bundle/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
    11: from /usr/local/bundle/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
    10: from /usr/local/bundle/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
     9: from /usr/local/bundle/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
     8: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/cli.rb:78:in `pull'
     7: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/cli.rb:41:in `handle_options'
     6: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/cli.rb:41:in `each'
     5: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/cli.rb:42:in `block in handle_options'
     4: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/cli.rb:79:in `block in pull'
     3: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/deployer/ssh.rb:48:in `pull_db'
     2: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/deployer/ssh/wpcli_sql_adapter.rb:34:in `adapt_remote_db!'
     1: from /usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/deployer/ssh.rb:79:in `download_remote_db'
/usr/local/bundle/gems/wordmove-4.0.1/lib/wordmove/deployer/ssh.rb:70:in `remote_run': Error code 1 returned by command "mysqldump --host=127.0.0.1 --port=3307 --user=xxx-xxx --password=xxx-xxx --result-file="/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql" wordpress": mysqldump: [Warning] Using a password on the command line interface can be insecure. (Wordmove::ShellCommandError)
mysqldump: Can't create/write to file '/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql' (Errcode: 13 - Permission denied)

Note: If I SSH into the remote machine as staging-user and run the following command: mysqldump --host=127.0.0.1 --port=3307 --user=xxx-xxx --password=xxx-xxx --result-file="/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql" wordpress I will also get: mysqldump: Can't create/write to file '/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql' (Errcode: 13 - Permission denied)

Now, If I run the command with sudo (still on the remote machine) sudo mysqldump --host=127.0.0.1 --port=3307 --user=xxx-xxx --password=xxx-xxx --result-file="/home/staging-user/wp-web/www/wp_files/wp-content/dump.sql" wordpress The backup is made fine, however the back up is owned by root (makes sense since I executed via sudo).

To also answer the other question:

from where the command ls-la

staging-user@do-xyz:~/wp-web/www/wp_files/wp-content$ ls -la
total 1064
drwxr-xr-x  6 www-data www-data    4096 Nov 18 11:48 .
drwxr-xr-x  5 www-data www-data    4096 Nov 18 12:42 ..
-rw-r--r--  1 root     root     1010268 Nov 18 12:43 dump.sql
-rw-r--r--  1 www-data www-data      28 Jan  9  2012 index.php
-rw-r--r--  1 www-data www-data   47766 Oct 18 11:38 object-cache.php
drwxr-xr-x 15 www-data www-data    4096 Nov 18 08:58 plugins
drwxr-xr-x  4 www-data www-data    4096 Nov 13 11:09 themes
drwxr-xr-x  2 www-data www-data    4096 Nov 13 11:09 upgrade
drwxr-xr-x  7 www-data www-data    4096 Sep 27 19:43 uploads

Please let me know if you have any other questions! Thank you for your help and this tool.

Question: I have read a few solutions, e.g. changing the ownership of the wp-content file (which also seems discouraged), or adding my ssh user to the www-data group. Are any of these suitable solutions from a security perspective? OR Is is possible for me to define somehow where the dump.sql gets stored on the remote machine, similar to how the full path of wordpress is required? This way I could have the dump.sql stored in a place that the ssh user has access to.

Also: I've slightly updated the movefile, and the items I've changed when pasting it here for privacy are:

local database password
staging vhost address to: test.xyz.com
wordpress_path, I replaced the Username (which I SSH into the virtual server) to staging-user /home/staging-user/wp-web/www/wp_files/
ssh host changed to do-xyz
removed the exclude for brevity here

--- Movefile---

global:
  sql_adapter: wpcli

local:
  vhost: http://localhost
  wordpress_path: /www/ # use an absolute path here

  database:
    name: "wordpress"
    user: "root"
    password: "some-password"
    host: "db" # -- DONT CHANGE --
    port: "3306"
    charset: "utf8mb4"

staging:
  vhost: https://test.xyz.com
  wordpress_path: /home/staging-user/wp-web/www/wp_files/ 

  database:
    name: wordpress
    user: "<%= ENV['STAGING_DB_USER'] %>"
    password: "<%= ENV['STAGING_DB_PASS'] %>"
    host: "127.0.0.1"
    port: 3307 
    charset: "utf8mb4"

  ssh:
    host: do-xyz

alessandro-fazzi commented 4 years ago

Really nice follow up, thank you.

My idea is that the problem is quite simple: you're connecting to the host with a user who has not write access to a directory of your web-app. Full stop. 😄

Is is possible for me to define somehow where the dump.sql gets stored on the remote machine

It is not. We assume you work on your remote host with a user enabled to take actions on files/folders inside the declared wordpress_path and I'd say that is ok to have such an assumption. (Considering too that the solution was thought to cover both SSH and FTP - jailed - needs)

Now, If I run the command with sudo (still on the remote machine) [...]

Actually as expected; moreover, that file is a temporary file and it will be deleted just after downloaded.

I have read a few solutions, e.g. changing the ownership of the wp-content file (which also seems discouraged)

I'd confirm: discouraged. Permissions are part of your web application so it's always better to leave them alone.

or adding my ssh user to the www-data group

I work on scenarios where the solution is this one, said that you will have also to tune group permissions adding write for the group. I consider this one a safe scenario. You could even directly use the www-data user, but it gets more complicated and a bit riskier.

Another take would be to directly use root and ensure permissions are not messed up after the push/pull with a remote hook.

As a final note: you'd have this problem even if you were not using wordmove but an rsync script. I mean it's a normal problem overall and nothing magic happening. And this is nice since I'm sure you can find a solution which best fit your needs and since you can search for solutions to this problem on the web ignoring the "wordmove" keyword :)

Let me know if you need further help/discussion.

Best luck

alessandro-fazzi commented 4 years ago

Closing for stall. But feel free to reopen

MarkErik commented 4 years ago

Dear @pioneerskies apologies for the late reply, and thank you for the suggestions.

In the process of trying to get this working, after adding hooks as you suggested, I ran into additional errors and then other tasks popped up before I could finish, and I paused on this for a bit.

In case others are interested, what I ended up with was including the following lines in my movefile.yml:

hooks: pull: before:

command: 'echo mypassword | sudo -S chmod 775 wp-content' where: remote after:
command: 'echo mypassword | sudo -S chmod 755 wp-content' where: remote

Thank you!

welaika / wordmove

Question / Issue (Errcode: 13 "Permission denied") Can't pull database, error during the backup step #543