search

deployphp / deployer

The PHP deployment tool with support for popular frameworks out of the box
https://deployer.org
MIT License
10.6k stars 1.48k forks source link

Application stops at Git clone #1064

Closed oliverkuchies closed 7 years ago

oliverkuchies commented 7 years ago
Q A
Issue Type Bug
Deployer Version 4.2.1
Local Machine OS Windows (Running on Ubuntu 16 Laravel Homestead)
Remote Machine OS Ubuntu 14.04 Linux

Description

Upon using dep deploy, the program freezes at 'SSH multiplexing initialization'. I have tested the connection to Bitbucket on the remote server and it is successful without an issue. The issue occurs when run off the local server; it simply freezes. There is no log to indicate that anything else is happening but it has simply stopped there. The SSH key on the remote server (for GIT) has a password. I am new to this application and wanted to use it for deployment, but this issue is stopping me at this stage :( Would appreciate any help!

If you're reporting a bug, please include following information

Steps to reproduce

Run dep deploy

Content of deploy.php

<?php
namespace Deployer;
require 'recipe/laravel.php';

// Configuration

set('ssh_type', 'native');
set('ssh_multiplexing', true);

set('repository', 'git@bitbucket.org:oliverkucharzewski/ddd.git');
add('shared_files', []);
add('shared_dirs', ['www']);

//add('writable_dirs', []);

// Servers
server('production', 'some.server.com')
    ->user('root')
    ->identityFile('~/.ssh/bitbucket.pub', '~/.ssh/bitbucket', 'passwordhere')
    ->set('deploy_path', '/var/www/api_server')
    ->pty(true);

// Tasks

desc('Restart PHP-FPM service');
task('php-fpm:restart', function () {
    // The user must have rights for restart service
    // /etc/sudoers: username ALL=NOPASSWD:/bin/systemctl restart php-fpm.service
    run('sudo systemctl restart php-fpm.service');
});
after('deploy:symlink', 'php-fpm:restart');

// [Optional] if deploy fails automatically unlock.
after('deploy:failed', 'deploy:unlock');
// Migrate database before symlink new release.

before('deploy:symlink', 'artisan:migrate');

Output log

= vagrant@homestead:/Code/api_server$ sudo dep deploy ✔ Executing task deploy:prepare ✔ Executing task deploy:lock ✔ Executing task deploy:release ➤ Executing task deploy:update_code ^C vagrant@homestead:/Code/api_server$ sudo dep deploy:unlock ✔ Executing task deploy:unlock vagrant@homestead:/Code/api_server$ sudo dep deploy -vvv ➤ Executing task deploy:prepare [production] > echo $0 SSH multiplexing initialization [production] < bash [production] > if [ ! -d /var/www/api_server ]; then mkdir -p /var/www/api_server; fi SSH multiplexing initialization [production] > if [ ! -L /var/www/api_server/current ] && [ -d /var/www/api_server/current ]; then echo true; fi SSH multiplexing initialization [production] > cd /var/www/api_server && if [ ! -d .dep ]; then mkdir .dep; fi SSH multiplexing initialization [production] > cd /var/www/api_server && if [ ! -d releases ]; then mkdir releases; fi SSH multiplexing initialization [production] > cd /var/www/api_server && if [ ! -d shared ]; then mkdir shared; fi SSH multiplexing initialization • done on [production] ✔ Ok [3s 42ms] ➤ Executing task deploy:lock [production] > if [ -f /var/www/api_server/.dep/deploy.lock ]; then echo 'true'; fi SSH multiplexing initialization [production] > touch /var/www/api_server/.dep/deploy.lock SSH multiplexing initialization • done on [production] ✔ Ok [1s 69ms] ➤ Executing task deploy:release [production] > cd /var/www/api_server && (if [ -h release ]; then echo 'true'; fi) SSH multiplexing initialization [production] < true [production] > cd /var/www/api_server && (rm -rf "$(readlink release)") SSH multiplexing initialization [production] > cd /var/www/api_server && (rm release) SSH multiplexing initialization [production] > cd /var/www/api_server && ([ -d releases ] && [ "$(ls -A releases)" ] && echo "true" || echo "false") SSH multiplexing initialization [production] < false [production] > cd /var/www/api_server && (if [ -d /var/www/api_server/releases/1 ]; then echo 'true'; fi) SSH multiplexing initialization [production] > cd /var/www/api_server && (date +"%Y%m%d%H%M%S") SSH multiplexing initialization [production] < 20170307081504 [production] > cd /var/www/api_server && (echo '20170307081504,1' >> .dep/releases) SSH multiplexing initialization [production] > cd /var/www/api_server && (mkdir /var/www/api_server/releases/1) SSH multiplexing initialization [production] > cd /var/www/api_server && (if [[ "$(man ln)" =~ "--relative" ]]; then echo "true"; fi) SSH multiplexing initialization [production] < true [production] > cd /var/www/api_server && (ln -nfs --relative /var/www/api_server/releases/1 /var/www/api_server/release) SSH multiplexing initialization • done on [production] ✔ Ok [5s 277ms] ➤ Executing task deploy:update_code [production] > which git SSH multiplexing initialization [production] < /usr/bin/git [production] > /usr/bin/git version SSH multiplexing initialization [production] < git version 2.7.4 [production] > if [ -h /var/www/api_server/release ]; then echo 'true'; fi SSH multiplexing initialization [production] < true [production] > readlink /var/www/api_server/release SSH multiplexing initialization [production] < releases/1 [production] > /usr/bin/git clone --recursive -q git@bitbucket.org:oliverkucharzewski/ddd.git /var/www/api_server/releases/1 2>&1 SSH multiplexing initialization

antonmedv commented 7 years ago

I think problem is with password for git. Try set pry(false) snd deploy again.

oliverkuchies commented 7 years ago

Hi @elfet , Thank you so much for your prompt reply - Here is the error I receive when turning pry to false. Please note I have tested the SSH connection to Github from the remote's end and there is no problem. - the key used has no problem by running the git clone comment and clone is completed successfully after password prompt.

Also; may I ask what pry() does?

Many thanks :) +1

 dep deploy -vvv
➤ Executing task deploy:prepare
[production] > echo $0
[production] < bash
[production] > if [ ! -d /var/www/api_server ]; then mkdir -p /var/www/api_server; fi
[production] > if [ ! -L /var/www/api_server/current ] && [ -d /var/www/api_server/current ]; then echo true; fi
[production] > cd /var/www/api_server && if [ ! -d .dep ]; then mkdir .dep; fi
[production] > cd /var/www/api_server && if [ ! -d releases ]; then mkdir releases; fi
[production] > cd /var/www/api_server && if [ ! -d shared ]; then mkdir shared; fi
• done on [production]
✔ Ok [1s 513ms]
➤ Executing task deploy:lock
[production] > if [ -f /var/www/api_server/.dep/deploy.lock ]; then echo 'true'; fi
[production] > touch /var/www/api_server/.dep/deploy.lock
• done on [production]
✔ Ok [531ms]
➤ Executing task deploy:release
[production] > cd /var/www/api_server && (if [ -h release ]; then echo 'true'; fi)
[production] < true
[production] > cd /var/www/api_server && (rm -rf "$(readlink release)")
[production] > cd /var/www/api_server && (rm release)
[production] > cd /var/www/api_server && ([ -d releases ] && [ "$(ls -A releases)" ] && echo "true" || echo "false")
[production] < false
[production] > cd /var/www/api_server && (if [ -d /var/www/api_server/releases/1 ]; then echo 'true'; fi)
[production] > cd /var/www/api_server && (date +"%Y%m%d%H%M%S")
[production] < 20170307092644
[production] > cd /var/www/api_server && (echo '20170307092644,1' >> .dep/releases)
[production] > cd /var/www/api_server && (mkdir /var/www/api_server/releases/1)
[production] > cd /var/www/api_server && (if [[ "$(man ln)" =~ "--relative" ]]; then echo "true"; fi)
[production] < true
[production] > cd /var/www/api_server && (ln -nfs --relative /var/www/api_server/releases/1 /var/www/api_server/release)
• done on [production]
✔ Ok [3s 520ms]
➤ Executing task deploy:update_code
[production] > which git
[production] < /usr/bin/git
[production] > /usr/bin/git version
[production] < git version 2.7.4
[production] > if [ -h /var/www/api_server/release ]; then echo 'true'; fi
[production] < true
[production] > readlink /var/www/api_server/release
[production] < releases/1
[production] > /usr/bin/git clone   --recursive -q git@bitbucket.org:oliverkucharzewski/ddd.git /var/www/api_server/releases/1 2>&1
➤ Executing task deploy:failed
• done on [production]
✔ Ok [0ms]
➤ Executing task deploy:unlock
[production] > rm -f /var/www/api_server/.dep/deploy.lock
• done on [production]
✔ Ok [252ms]
[Symfony\Component\Process\Exception\ProcessFailedException]                
The command "ssh -A -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChe  
cking=no -o ControlMaster=auto -o ControlPersist=5 -o ControlPath='~/.ssh/  
deployer_mux_root@manage.ddd.com:22' -p '22' -i '/home/vagrant  
/.ssh/bitbucket' 'root@manage.ddd.com' '/usr/bin/git clone   -  
  -recursive -q git@bitbucket.org:oliverkucharzewski/ddd.git /var/www/api_server/releases/1 2>&1'" failed.                      
  Exit Code: 128(Invalid exit argument)                                       
  Working directory: /home/vagrant/Code/api_server                            
  Output:                                                                     
  ================                                                            
  Permission denied (publickey).                                              
  fatal: Could not read from remote repository.                               
  Please make sure you have the correct access rights                         
  and the repository exists.                                                  
  Error Output:                                                               
  ================                                                            

Exception trace:
 () at phar:///usr/local/bin/dep/vendor/symfony/process/Process.php:233
 Symfony\Component\Process\Process->mustRun() at phar:///usr/local/bin/dep/src/Server/Remote/NativeSsh.php:87
 Deployer\Server\Remote\NativeSsh->run() at phar:///usr/local/bin/dep/src/functions.php:310
 Deployer\run() at phar:///usr/local/bin/dep/recipe/deploy/update_code.php:59
 Deployer\{closure}() at n/a:n/a
 call_user_func() at phar:///usr/local/bin/dep/src/Task/Task.php:85
 Deployer\Task\Task->run() at phar:///usr/local/bin/dep/src/Executor/SeriesExecutor.php:40
 Deployer\Executor\SeriesExecutor->run() at phar:///usr/local/bin/dep/src/Console/TaskCommand.php:95
 Deployer\Console\TaskCommand->execute() at phar:///usr/local/bin/dep/vendor/symfony/console/Command/Command.php:262
 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:826
 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:189
 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/dep/vendor/symfony/console/Application.php:120
 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/dep/src/Deployer.php:186
 Deployer\Deployer->run() at phar:///usr/local/bin/dep/bin/dep:113
 require() at /usr/local/bin/dep:4

deploy [-p|--parallel]
antonmedv commented 7 years ago

pty(true); forces ssh to allocate terminal. Usually this is a bad think and you not want to do this, use non-interactive mode instead. Btw, this will be deprecated in next v5 and only will be allowed from run func.

Try disable ssh multiplexing:

set('ssh_multiplexing', false);
antonmedv commented 7 years ago

Maybe there is bug with it.

oliverkuchies commented 7 years ago

Can confirm with multiplexing set to false the issue still exists. Would appreciate a fix for this & will shout you a drink for your time :) Really excited to use this platform!

antonmedv commented 7 years ago

Please, try run this:

$ ssh-add /path/to/keys
$ dep deploy
oliverkuchies commented 7 years ago

Hi @elfet - Still no luck. The same error occurs, no change.

Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Error Output:

I tried testing the SSH connection from the remote server and it was successful . Please note; from dep deploy: i'm never asked for the password for the private key associated with the connection for GIT so it feels like its never being unlocked. Here is the log confirming that ssh works from remote server (also works on local if neccessary): root@vs-oliverolidev-me:~/.ssh# ssh git@bitbucket.com PTY allocation request failed on channel 0 logged in as oliverkucharzewski. You can use git or hg to connect to Bitbucket. Shell access is disabled. Connection to bitbucket.com closed.

antonmedv commented 7 years ago

@oliverkuchies please, tell me more about you configuration. I will try to implement same environment.

You have one key on local machine and using agent forward to pull on remove? Do you have passphrase? Or do you use deploy keys from bitbucket? Tell me more so i can recreate your situation locally.

oliverkuchies commented 7 years ago

Hi Elfet - This is how its configured:

Local machine: Local machine is where I edit all my files; I was hoping to deploy them to the remote machine. Local machine is where I run deployer - this is running on a Vagrant VM with Ubuntu.
To connect to the remote machine from the local machine; I use a private key which is authorised on the remote server. The key is password protected. (not familiar with agent forward term) Running on Laravel Homestead (latest)

Remote Server: When using the remote server; I use PuTTY to connect to it directly and run scripts. When i demonstrated a successful connection to Git; it was from this server. This requires that key authentication. This server is running Ubuntu 16 and is hosted in a remote VPS server.

Bitbucket: Bitbucket is connected to from my remote server (which is where the git pull transaction takes place it seems); I have a private key stored called id_rsa which connects to it. This key is authorised with a public key on the Bitbucket server. The id_rsa key has a password.

When connecting to Bitbucket from the remote server; I am prompted for a password. When using deployer; I am not prompted for a password (which is why i think it doesn't go through)

Hopefully this is sufficient; please let me know if i've missed something.

Thank you Elfet!

antonmedv commented 7 years ago

When connecting to Bitbucket from the remote server; I am prompted for a password.

Deployer can't ask for git passwords (for now, i have some thoughts on this for next release). Try create key what does not have a password for bitbucket and then try to deploy. I think it can help.

oliverkuchies commented 7 years ago

Hi Anton! This was indeed the case. It seem deployer does not support the passwords. Passwordless works fine. Thanks

antonmedv commented 7 years ago

Passwords for git will be in next release. See #1092

nsteinfeld commented 6 years ago

I am a total newbie to Symfony and Deployer and have the same problem as described by the poster. I run Deployer 4.3 because my local machine only runs up to PHP 5.6. I think I created a KeyGen without a password (could you list the steps?). I am running Deployer from my local machine and am trying to connect to both the server and the git repository to deploy from my local machine to the shared server (Apache). 

// Configuration
set('repository', 'git@github.com:user/project.git');
set('ssh_type', 'native');
set('ssh_multiplexing', true);

set('env', 'prod');
//set('default_stage', 'prod');

add('shared_files', ['web/.htaccess', 'app/config/configuration.yml', 'web/robots.txt']);
add('shared_dirs', []);

add('writable_dirs', []);

// Servers
server('dev','server')
    ->user('ns7637') // Defind SSH username
    ->identityFile('SSHkeyfile', '~/SSHkeyfile')
    ->set('deploy_path', '/path/to/server/directory') // Define the base path to deploy your project to.
    ->set('branch', 'master')
    ->stage('default_stage', 'staging');

Here is the error I get:

  The command "ssh -A -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ControlMaster=auto -o ControlPersist=5 -o ControlPath='~/.ssh/deployer_mux_user@server:22' -p '22' -i ‘/path/toSSHKey/key’ -t 'ns7637@file.laits.utexas.edu' bash -s" failed.  

  Exit Code: 128(Invalid exit argument)                                                                                                                                                                                                                                                     

  Working directory: /path/to/SymfonyDirectory/                                                                                                                                                                                                                                                    

  Output:                                                                                                                                                                                                                                                                                   
  ================                                                                                                                                                                                                                                                                          
  ssh: connect to host github.com port 22: Connection timed out                                                                                                                                                                                                                             
  fatal: The remote end hung up unexpectedly                                                                                                                                                                                                                                                

  Error Output:                                                                                                                                                                                                                                                                             
  ================

I was able to succesfully clone from the git directory using command line without inputting a password: $git clone -b master --depth 1 --recursive -q git@github.com:user/project.git test So the problem isn't the git connection. Does my ssh key need to be on the server where I am deploying to? Or what am I doing wrong?

antonmedv commented 6 years ago

Check agent forwarding.

antonmedv commented 6 years ago

And try to upgrade to v6.0.3

nsteinfeld commented 6 years ago

Thanks for for your answer. I cannot use deployer 5 or 6 because it requires PHP > 5.6. I run Deployer 4.3 on my local development environment and the remote (shared) server I am deploying to runs PHP 5.3 and will not upgrade PHP in the near future. I did add a ssh config and added Forward Agent to the server domain, but I run the following ssh test on the server while logged in, ssh -T git@github.com it times out. I am in contact with the people who maintain the server. What do I need to ask from them to get this to work?

antonmedv commented 6 years ago

Check if you can clone it using deploy key. Check what nothing prompts in tty mode.

nsteinfeld commented 6 years ago

So I switched from a SSH key to a deploy key and I used the -t option when connecting to the server. Or were you suggesting to add pty(true) to the deploy.php file? I tried this before and attempting a deployment failed immediately. Did I miss something you suggested? Any other suggestions?

antonmedv commented 6 years ago

Is it really difficult to upgrade php to 7?