search

sourcebroker / deployer-extended-database

Deployer tasks to manage database synchronization between application instances.
MIT License
39 stars 13 forks source link

db:download task : Read-only file system #13

Open tazeverywhere opened 2 years ago

tazeverywhere commented 2 years ago

Hi @kszymukowicz,

I use deployer-extended-wp-composer with a roots-bedrock setup for all my wp projects and usually everything works nicely. Dev-env is powered by local app (localwp.com).

Recently, after an upgrade from deployer-extended-wordpress-composer v4.0 to v7.0, during process of dep db:pull production, I have now an error on the "db:download" step.

As I understand the error, the process failed during the dumps trying to create a dir on the remote server ("mkdir" see error output above) ? I have a Read-only file system error.

But if I look at the dumps folder on the remote server, dumps are created. If I try a db:backup task, everything works as well.

Everything work fine for deploy, but I can't find the problem with db:pull.

Cmd and error output are:

dep db:pull production
➤ Executing task db:pull
In Process.php line 250:

The command "./vendor/bin/dep db:download production --options=dumpcode:e10709937a788678d7d9c0d2372b48fa " failed.                           

  Exit Code: 1(General error)                                                                                                                  

  Working directory: /Users/xxx/LocalWebsites/mywebsite/app                                                                                  

  Output:                                                                                                                                      
  ================                                                                                                                             
  ➤ Executing task db:download                                                                                                                 

  Error Output:                                                                                                                                
  ================                                                                                                                             

  In Process.php line 250:                                                                                                                     

    The command "[ -d /mywebsite/app/wp/.dep/database/dumps ] || mkdir -p /mywebsite/ap                                                                
    p/wp/.dep/database/dumps" failed.                                                                                                          

    Exit Code: 1(General error)                                                                                                                

    Working directory: /Users/xxx/LocalWebsites/mywebsite/app                                                                                

    Output:                                                                                                                                    
    ================                                                                                                                           

    Error Output:                                                                                                                              
    ================                                                                                                                           
    mkdir: /mywebsite/app/wp/.dep/database/dumps: **Read-only file system**

Any idea or advice ? Thanks in advance for your help, Sylvain

kszymukowicz commented 2 years ago

hi look what changed in version 6.0.0 https://github.com/sourcebroker/deployer-extended-wordpress-composer/blob/master/CHANGELOG.rst

The config is now read from config/.env. Did you modified your files for this?

Also try to run the command with -vvv. Maybe there will be more data to debug.

Could be that this folder is tried to be created locally because for some reason the instance name is wrongly read for local instance.

tazeverywhere commented 2 years ago

Hi,

Thanks for your answer!

Yes, .env is in config folder now as all other (breaking) changes after upgrade are made. Running dep db:pull production -vvv , I can see some reference to server=staging,why ?

➤ Executing task db:pull
[localhost] > ./vendor/bin/dep db:export production --options=dumpcode:ef2a5c4ed5eb5614ad525190c24bd954  -vvv
[localhost] < ➤ Executing task db:export
[localhost] < [production] > if [ -L /mywebsite/app/wp/release ]; then echo 'true'; fi
[localhost] < [production] < ssh multiplexing initialization
[localhost] < [production] < Linux myServer 4.19.0-17-amd64 #1 SMP Debian 4.19.194-3 (2021-07-18) x86_64
[localhost] < The programs included with the Debian GNU/Linux system are free software;
[localhost] < the exact distribution terms for each program are described in the
[localhost] < individual files in /usr/share/doc/*/copyright.
[localhost] < Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
[localhost] < permitted by applicable law.
[localhost] < [production] > if [ -L /mywebsite/app/wp/release ]; then echo 'true'; fi
[localhost] < [production] > if [ -e '/mywebsite/app/wp/current/vendor/bin/dep' ]; then echo 'true'; fi
[localhost] < [production] < true
[localhost] < [production] > cd /mywebsite/app/wp/current && /usr/bin/php7.3 /mywebsite/app/wp/current/vendor/bin/dep db:export --options=dumpcode:ef2a5c4ed5eb5614ad525190c24bd954  -vvv
[localhost] < [production] < ➤ Executing task db:export
[localhost] < [production] < [localhost] > command -v 'mysqldump' || which 'mysqldump' || type -p 'mysqldump'
[localhost] < [production] < [localhost] < /usr/bin/mysqldump
[localhost] < [production] < [localhost] > [ -d /mywebsite/app/wp/.dep/database/dumps ] || mkdir -p /mywebsite/app/wp/.dep/database/dumps
================= HERE =======================
[localhost] < [production] < [localhost] > export MYSQL_PWD='myPassword' && /usr/bin/mysqldump --no-data=true --default-character-set=utf8 --no-tablespaces -h'localhost' -P'3306' -u'mySqlUser' 'mySshUser' -r'/mywebsite/app/wp/.dep/database/dumps/2021-08-01_16-16-15#server=staging#dbcode=database_default#dumpcode=ef2a5c4ed5eb5614ad525190c24bd954#type=structure.sql'
[localhost] < [production] < [localhost] > export MYSQL_PWD='myPassword' && /usr/bin/mysqldump --opt --skip-lock-tables --single-transaction --no-create-db --default-character-set=utf8 --no-tablespaces -h'localhost' -P'3306' -u'mySqlUser' 'mySshUser' -r'/mywebsite/app/wp/.dep/database/dumps/2021-08-01_16-16-15#server=staging#dbcode=database_default#dumpcode=ef2a5c4ed5eb5614ad525190c24bd954#type=data.sql'
[localhost] < [production] < • done on [staging]
================= HERE =======================
[localhost] < [production] < ✔ Ok [226ms]
[localhost] < • done on [production]
[localhost] < ✔ Ok [1s 658ms]
[localhost] > ./vendor/bin/dep db:download production --options=dumpcode:ef2a5c4ed5eb5614ad525190c24bd954  -vvv
[localhost] < ➤ Executing task db:download
[localhost] < [production] > [ -d /mywebsite/app/wp/.dep/database/dumps ] || mkdir -p /mywebsite/app/wp/.dep/database/dumps
[localhost] < [localhost] > [ -d /mywebsite/app/wp/.dep/database/dumps ] || mkdir -p /mywebsite/app/wp/.dep/database/dumps
[localhost] < [localhost] < mkdir: /mywebsite/app/wp/.dep/database/dumps: Read-only file system

Exception trace:

  at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/vendor/symfony/process/Process.php:250
 Symfony\Component\Process\Process->mustRun() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Utility/ProcessRunner.php:56
 Deployer\Utility\ProcessRunner->run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/functions.php:329
 Deployer\runLocally() at /Users/xxx/LocalWebsites/myWebsite/app/vendor/sourcebroker/deployer-extended-database/deployer/db/task/db_pull.php:46
 SourceBroker\DeployerLoader\Utility\FileUtility->Deployer\{closure}() at n/a:n/a
 call_user_func() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Task/Task.php:105
 Deployer\Task\Task->run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Executor/SeriesExecutor.php:60
 Deployer\Executor\SeriesExecutor->run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Console/TaskCommand.php:144
 Deployer\Console\TaskCommand->execute() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:924
 Symfony\Component\Console\Application->doRunCommand() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Console/Application.php:133
 Deployer\Console\Application->doRunCommand() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:265
 Symfony\Component\Console\Application->doRun() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/vendor/symfony/console/Application.php:141
 Symfony\Component\Console\Application->run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/src/Deployer.php:326
 Deployer\Deployer::run() at phar:///Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep/bin/dep:136
 require() at /Users/xxx/LocalWebsites/myWebsite/app/vendor/deployer/dist/dep:4

The only thing I see is that I needed to customize the instances names, because of your default changes, and re-set it with set('instance_...). BUT, I also have a "staging/test" instance, and the only way I found to be regognized by deployer is fallback it to default_stage (see above). I don't think it's the right way and maybe it throw an error somewhere ? (I didn't found a clue in your code :P )

namespace Deployer;

require __DIR__ . '/vendor/autoload.php';
new \SourceBroker\DeployerExtendedWordpressComposer\Loader();

set('instance_local_name', 'development');
set('instance_live_name', 'production');
set('default_stage', 'staging');

#(...)

If I run dep config:hosts it says

+-------------+-------------------------+-------------+-------+------------------------------------------+
| Host        | Hostname                | Stage       | Roles | Deploy path                              |
+-------------+-------------------------+-------------+-------+------------------------------------------+
| production  | prod.mywebsite.ch       | production  |       | /mywebsite/app/wp                        |
| staging     | stage.mywebsite.ch      | staging     |       | /mywebsite/app/wp                        |
| development |                         | development |       | /Users/xxx/LocalWebsites/mywebsite/app   |
+-------------+-------------------------+-------------+-------+------------------------------------------+
Ok.

Thanks again for your time.

kszymukowicz commented 2 years ago

You should not touch set('default_stage' ..); default_stage is set dynamically based on WP_ENV or WP_INSTANCE.

kszymukowicz commented 2 years ago

What is the behaviour when you remove set('default_stage', 'staging'); ?

tazeverywhere commented 2 years ago

Ok I didn't know that. So I removed the "set default_instance". I have now a Hostname or stage "dev" not found. error (indeed, that's why I did the default_stage thing..), even with a dep deploy xxx.

 In Client.php line 103:
 [Deployer\Exception\RuntimeException (1)]                                                              
  The command "cd /mywebsite/app/wp/release && /usr/bin/php7.3 /mywebsite/app/wp/release/vendor/bin/dep db:expo  
  rt --options=dumpcode:backup_for_release_7_a85a7cf24661364805dde6058153e9d3  -vvv" failed.             

  Exit Code: 1 (General error)                                                                           

  Host Name: production                                                                                  

  ================                                                                                       

  In HostSelector.php line 59:                                                                           

    [Deployer\Exception\Exception]                                                                       
    Hostname or stage `dev` was not found.                                                               

  Exception trace:                                                                                       
    at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist/dep/src/Host/HostSelector.php:59             
   Deployer\Host\HostSelector->getHosts() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist/dep/s  
  rc/Console/TaskCommand.php:120                                                                         
   Deployer\Console\TaskCommand->execute() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist/dep/  
  vendor/symfony/console/Command/Command.php:255                                                         
   Symfony\Component\Console\Command\Command->run() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/  
  dist/dep/vendor/symfony/console/Application.php:924                                                    
   Symfony\Component\Console\Application->doRunCommand() at phar:///mywebsite/app/wp/releases/7/vendor/depl  
  oyer/dist/dep/src/Console/Application.php:133                                                          
   Deployer\Console\Application->doRunCommand() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist  
  /dep/vendor/symfony/console/Application.php:265                                                        
   Symfony\Component\Console\Application->doRun() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/di  
  st/dep/vendor/symfony/console/Application.php:141                                                      
   Symfony\Component\Console\Application->run() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist  
  /dep/src/Deployer.php:326                                                                              
   Deployer\Deployer::run() at phar:///mywebsite/app/wp/releases/7/vendor/deployer/dist/dep/bin/dep:136      
   require() at /mywebsite/app/wp/releases/7/vendor/deployer/dist/dep:4                                      

  db:export [-p|--parallel] [-l|--limit

a dep config:hosts say the same

In HostSelector.php line 59:

  Hostname or stage `dev` was not found.           

config:hosts [-p|--parallel] [-l|--limit LIMIT] [--no-hooks] [--log LOG] [--roles ROLES] [--hosts HOSTS] [-o|--option OPTION] [--] [<stage>]
tazeverywhere commented 2 years ago

if i add a WP_INSTANCE in .env, in addition of INSTANCE.. IT WORKS !! (same for production and staging)

INSTANCE='development'
WP_INSTANCE='development'

It's strange, because WP_ENV is already defined in .env (and works in Bedrock), and as I can see in EnvDriver.php, it should also works, no ?