DeployHubProject / DeployHub-Pro

DeployHub Pro Pipeline Status Project
https://www.openmakesoftware.com/application-release-automation-for-continuous-delivery/
Other
11 stars 4 forks source link

Windows IIS Example #262

Closed sbtaylor15 closed 4 years ago

sbtaylor15 commented 5 years ago

Start with basic start/stop website

sbtaylor15 commented 5 years ago

@piyush94 - what restart process do you want to use?

  1. Send files to server in staging area
  2. Stop Website
  3. Copy files from staging to web root
  4. Start Website

Or

  1. Send files to server in staging area
  2. Copy files from staging to web root
  3. Stop Website
  4. Start Website

CC: @svisagan83

sbtaylor15 commented 5 years ago

First process for restart

sbtaylor15 commented 5 years ago

@piyush94 - Please pull the latest docker image. Also, here are the procedures to import into your database. Goto the Functions & Procedures Explorer tab -> choose GLOBAL -> right click "Import a Function or Procedure into this Domain"

procs.zip

Next, configure the RunAnsibleAction as follows.

action gitcheckout writeenv runplaybook

Next configure the Component Attributes. Don't forget to setup a Component Item for the site.zip

compattrs

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 It's working.

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 Just one thing, i added one more playbook to add windows firewall rule. It works if it is the single component inside the application but gives error if i add the above IIS playbook as well in the application flow.

image

image

INFO: Starting deployment #182
--
INFO: Deploying Application "Deploy_IIS_Site" into Environment "IIS_Env"
INFO: Deploying Component IIS_Test_Site
INFO: Checked out site.zip;1
INFO: "IIS_Test_Site" - Item "site zip": Checked out 1 file(s) from repository "ftp-repo"
INFO: Deploying Component IIS_Test_Site to Server inblr-uvm-1118
INFO: Deployed File site.zip -> inblr-uvm-1118:c:\staging\site.zip
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying IIS_Test_Site to inblr-uvm-1118
PLAY [IIS Update] **************************************************************TASK [create new website's directory] ******************************************ok: [inblr-uvm-1118.eu.uis.unisys.com]TASK [stop website] ************************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [unzip code to site path] *************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [start website] ***********************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]PLAY RECAP *********************************************************************inblr-uvm-1118.eu.uis.unisys.com : ok=4    changed=3    unreachable=0    failed=0
INFO: Deploying Component WIN_Firewall using Custom Action WIN_Firewall_Rule_Action
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying WIN_Firewall to inblr-uvm-1118
PLAY [Add/Update Windows Firewall Rule] ****************************************ERROR! Syntax Error while loading YAML.  found unknown escape characterThe error appears to have been in '/tmp/tmpe3t55r87/group_vars/all.yml': line 15, column 18, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:component: WIN_Firewalldep.filelist: "c:\staging/site.zip"                 ^ hereThis one looks easy to fix.  It seems that there is a value startedwith a quote, and the YAML parser is expecting to see the line endedwith the same kind of quote.  For instance:    when: "ok" in result.stdoutCould be written as:   when: '"ok" in result.stdout'Or equivalently:   when: "'ok' in result.stdout"
INFO: Return code is 4
Exception at line 0: Custom Action WIN_Firewall_Rule_Action for Component WIN_Firewall Failed: Return Code: 4
Stacktrace: server inblr-uvm-1118 dropzone 3 deploy  component WIN_Firewall application Deploy_IIS_Site avloop application Deploy_IIS_Site environment IIS_Env global
INFO: Time taken 0.053048 seconds

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 Playbook:

https://github.com/piyush94/ansible/blob/master/firewall_rule.yml

sbtaylor15 commented 5 years ago

Did you set the Action as a Post Action on the Application or on the Component?

piyush94 commented 5 years ago

@sbtaylor15 First time i set as a Post Action and nothing happened as i guess because the firewall_rule component has no component item. Second time i set as Custom Action and the error came. I also tried creating new Action object for firewall_rule component with same flow but got the same error.

CC: @svisagan83

sbtaylor15 commented 5 years ago

@piyush94 That is correct the Post Action won't get trigger when there are no component items. So the Custom Action is the way to go. Add an attribute to the Application Version called debug_ansible and set the value = 1. This will keep the ansible files around after the deployment. The tmp directory for the ansible files will be displayed in the output. Send us a zip of those files.

piyush94 commented 5 years ago

@sbtaylor15 zip file: ansible_debug.zip

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 It's also giving error if i try to deploy single IIS component to multiple endpoints.

INFO: Starting deployment #187
--
INFO: Deploying Application "Deploy_IIS_Site" into Environment "IIS_Env"
INFO: Deploying Component IIS_Test_Site
INFO: Checked out site.zip;1
INFO: "IIS_Test_Site" - Item "site zip": Checked out 1 file(s) from repository "ftp-repo"
INFO: Starting Sequential Deployment of Component IIS_Test_Site
INFO: Deploying Component IIS_Test_Site to Server inblr-uvm-1118
INFO: Deployed File site.zip -> inblr-uvm-1118:c:\staging\site.zip
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying IIS_Test_Site to inblr-uvm-1118
PLAY [IIS Update] **************************************************************TASK [create new website's directory] ******************************************ok: [inblr-uvm-1118.eu.uis.unisys.com]TASK [stop website] ************************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [unzip code to site path] *************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [start website] ***********************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]PLAY RECAP *********************************************************************inblr-uvm-1118.eu.uis.unisys.com : ok=4    changed=3    unreachable=0    failed=0   /tmp/tmp2uop7vne
INFO: Deploying Component IIS_Test_Site to Server USTR-UVM-15937
INFO: Deployed File LICENSE -> USTR-UVM-15937:c:\staging\LICENSE
INFO: Deployed File README.md -> USTR-UVM-15937:c:\staging\README.md
INFO: Deployed File id.yml -> USTR-UVM-15937:c:\staging\id.yml
INFO: Deployed File iis.yml -> USTR-UVM-15937:c:\staging\iis.yml
INFO: Deployed File mkdir.yml -> USTR-UVM-15937:c:\staging\mkdir.yml
INFO: Deployed File nsupdate.yml -> USTR-UVM-15937:c:\staging\nsupdate.yml
INFO: Deployed File env.dat -> USTR-UVM-15937:c:\staging\env.dat
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying IIS_Test_Site to USTR-UVM-15937
PLAY [IIS Update] **************************************************************ERROR! Syntax Error while loading YAML.  found unknown escape characterThe error appears to have been in '/tmp/tmp0q9sdh9m/group_vars/all.yml': line 16, column 18, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:debug_ansible: 1dep.filelist: "c:\staging/site.zip"                 ^ hereThis one looks easy to fix.  It seems that there is a value startedwith a quote, and the YAML parser is expecting to see the line endedwith the same kind of quote.  For instance:    when: "ok" in result.stdoutCould be written as:   when: '"ok" in result.stdout'Or equivalently:   when: "'ok' in result.stdout"/tmp/tmp0q9sdh9m
INFO: Completed Parallel Deployment
INFO: Time taken 0.051058 seconds

zipfile: iis_debug.zip

CC: @svisagan83

sbtaylor15 commented 5 years ago

@piyush94 the latest image has this fix for this. It will change the dep.filelist to a valid yaml key as dep_filelist

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 still not working.. when deploying to second endpoint it also starts to copy the playbook files, and still the above error is coming.

INFO: Starting deployment #266
--
INFO: Deploying Application "Deploy_IIS_Site" into Environment "IIS_Env"
INFO: Deploying Component IIS_Test_Site
INFO: Checked out site.zip;1
INFO: "IIS_Test_Site" - Item "site zip": Checked out 1 file(s) from repository "ftp-repo"
INFO: Starting Sequential Deployment of Component IIS_Test_Site
INFO: Deploying Component IIS_Test_Site to Server inblr-uvm-1118
INFO: Deployed File site.zip -> inblr-uvm-1118:c:\staging\site.zip
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying IIS_Test_Site to inblr-uvm-1118
PLAY [IIS Update] **************************************************************TASK [create new website's directory] ******************************************ok: [inblr-uvm-1118.eu.uis.unisys.com]TASK [stop website] ************************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [unzip code to site path] *************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [start website] ***********************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]PLAY RECAP *********************************************************************inblr-uvm-1118.eu.uis.unisys.com : ok=4    changed=3    unreachable=0    failed=0   /tmp/tmplqdoqlvs
INFO: Deploying Component IIS_Test_Site to Server USTR-UVM-15937
INFO: Deployed File LICENSE -> USTR-UVM-15937:c:\staging\LICENSE
INFO: Deployed File README.md -> USTR-UVM-15937:c:\staging\README.md
INFO: Deployed File id.yml -> USTR-UVM-15937:c:\staging\id.yml
INFO: Deployed File iis.yml -> USTR-UVM-15937:c:\staging\iis.yml
INFO: Deployed File mkdir.yml -> USTR-UVM-15937:c:\staging\mkdir.yml
INFO: Deployed File nsupdate.yml -> USTR-UVM-15937:c:\staging\nsupdate.yml
INFO: Deployed File env.dat -> USTR-UVM-15937:c:\staging\env.dat
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying IIS_Test_Site to USTR-UVM-15937
PLAY [IIS Update] **************************************************************ERROR! Syntax Error while loading YAML.  found unknown escape characterThe error appears to have been in '/tmp/tmpib0w0pvl/group_vars/all.yml': line 16, column 18, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:debug_ansible: 1dep_filelist: "c:\staging/site.zip"                 ^ hereThis one looks easy to fix.  It seems that there is a value startedwith a quote, and the YAML parser is expecting to see the line endedwith the same kind of quote.  For instance:    when: "ok" in result.stdoutCould be written as:   when: '"ok" in result.stdout'Or equivalently:   when: "'ok' in result.stdout"/tmp/tmpib0w0pvl
INFO: Completed Parallel Deployment
INFO: Starting Sequential Deployment of Component WIN_Firewall
INFO: Deploying Component WIN_Firewall using Custom Action WIN_Firewall_Rule_Action
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying WIN_Firewall to inblr-uvm-1118
PLAY [Add/Update Windows Firewall Rule] ****************************************ERROR! Syntax Error while loading YAML.  found unknown escape characterThe error appears to have been in '/tmp/tmpmvmvz_98/group_vars/all.yml': line 16, column 18, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:debug_ansible: 1dep_filelist: "c:\staging/site.zip"                 ^ hereThis one looks easy to fix.  It seems that there is a value startedwith a quote, and the YAML parser is expecting to see the line endedwith the same kind of quote.  For instance:    when: "ok" in result.stdoutCould be written as:   when: '"ok" in result.stdout'Or equivalently:   when: "'ok' in result.stdout"/tmp/tmpmvmvz_98
INFO: Return code is 4
INFO: Deploying Component WIN_Firewall using Custom Action WIN_Firewall_Rule_Action
Cloning into '.'...
Your branch is up to date with 'origin/master'.
Deploying WIN_Firewall to USTR-UVM-15937
PLAY [Add/Update Windows Firewall Rule] ****************************************ERROR! Syntax Error while loading YAML.  found unknown escape characterThe error appears to have been in '/tmp/tmpu8lz8bi3/group_vars/all.yml': line 16, column 18, but maybe elsewhere in the file depending on the exact syntax problem.The offending line appears to be:debug_ansible: 1dep_filelist: "c:\staging/LICENSE" "c:\staging/README.md" "c:\staging/id.yml" "c:\staging/iis.yml" "c:\staging/mkdir.yml" "c:\staging/nsupdate.yml" "c:\staging/env.dat"                 ^ hereThis one looks easy to fix.  It seems that there is a value startedwith a quote, and the YAML parser is expecting to see the line endedwith the same kind of quote.  For instance:    when: "ok" in result.stdoutCould be written as:   when: '"ok" in result.stdout'Or equivalently:   when: "'ok' in result.stdout"We could be wrong, but this one looks like it might be an issue withunbalanced quotes.  If starting a value with a quote, make sure theline ends with the same set of quotes.  For instance this arbitraryexample:    foo: "bad" "wolf"Could be written as:    foo: '"bad" "wolf"'/tmp/tmpu8lz8bi3
INFO: Return code is 4
Exception at line 0: Custom Action WIN_Firewall_Rule_Action for Component WIN_Firewall Failed: Return Code: 4
Stacktrace: server inblr-uvm-1118 dropzone 4 deploy  component WIN_Firewall application Deploy_IIS_Site avloop application Deploy_IIS_Site environment IIS_Env global
INFO: Time taken 0.084084 seconds

CC: @svisagan83

sbtaylor15 commented 5 years ago

@piyush94 - We reworked the logic to fix a clash at the dropzone level. The yaml syntax error and extra files being deployed is fixed.

Here is the new WriteEnv2File proc code:

  set -g RspFile = ${dropzone.path} + "/env.dat";
    set d = now();
    set sname = "s" + ${d.to_int()};

    eval("using stream \$$sname;");
    eval("set envstream = \$$sname;");

    using stream $envstream {
    echo "dropzone: " + ${dropzone.path};
    echo "environment: " + ${environment.name};
    echo "application: " + ${application.name};
    echo "component: " + ${component.name};
    echo "server_name: " + ${server.name};
    echo "targetOS: " + ${server.type};

    set sc = ${server.credential};
    if ($sc != null)
    {
     echo "sshuser: " + ${sc.username};
     echo "sshpass: " + ${sc.password};
    }
    echo "server_hostname: " + ${server.hostname};
    echo "component: " + ${component.name};

    if (${environment.id})
    {    
     foreach(envat: ${environment.attributes}) {  
       echo $envat + ": " + ${environment.attributes[$envat]};
     }
     set creds = ${environment.attributes['creds']};

     if ($creds)
     {       
      foreach(cred: $creds) {
       set credname = ${creds[$cred]};
       set c = getcredential($credname);

       echo $cred + "user: " + ${c.username};
       echo $cred + "pass: " + ${c.password}; 
      }
     }  
    }

    if (${application.id})
    {   
     foreach(appat: ${application.attributes}) {  
       echo $appat + ": " + ${application.attributes[$appat]};
     }

     set creds = ${application.attributes['creds']};
     if ($creds)
     {  
      foreach(cred: $creds) {
       set credname = ${creds[$cred]};
       set c = getcredential($credname);

       echo $cred + "user: " + ${c.username};
       echo $cred + "pass: " + ${c.password}; 
      }
     }
    } 

    if (${server.id})
    { 
     foreach(srvat: ${server.attributes}) {  
      echo $srvat + ": " + ${server.attributes[$srvat]};
     }

     set creds = ${server.attributes['creds']};
     if ($creds)
     {  
      foreach(cred: $creds) {
       set credname = ${creds[$cred]};
       set c = getcredential($credname);

       echo $cred + "user: " + ${c.username};
       echo $cred + "pass: " + ${c.password}; 
      } 
     }
    } 

    if (${component.id})
    {                       
     foreach(compat: ${component.attributes}) {  
      echo $compat + ": " + ${component.attributes[$compat]};       
     }

     set creds = ${component.attributes['creds']};
     if ($creds)
     {
      foreach(cred: $creds) {
       set credname = ${creds[$cred]};
       set c = getcredential($credname);

       echo $cred + "user: " + ${c.username};
       echo $cred + "pass: " + ${c.password}; 
      }
     }
    }

    create(file: "env.dat", stream: $envstream);
 }

Also, the component attributes have changed. The git clone is no longer used. Instead, the iss.yml is read over http.

Here are the component attributes comp-attrs

Here is the new AnsiblePlaybook Action ansible-action

Please pull the latest docker image

CC: @svisagan83

piyush94 commented 5 years ago

@sbtaylor15 It's not copying files to second endpoint.

INFO: Starting deployment #276
--
INFO: Deploying Application "Deploy_IIS_Site" into Environment "IIS_Env"
INFO: Deploying Component IIS_Test_Site
INFO: Checked out site.zip;1
INFO: "IIS_Test_Site" - Item "site zip": Checked out 1 file(s) from repository "ftp-repo"
INFO: Starting Sequential Deployment of Component IIS_Test_Site
INFO: Deploying Component IIS_Test_Site to Server inblr-uvm-1118
INFO: Deployed File site.zip -> inblr-uvm-1118:c:\staging\site.zip
fatal: repository '.' does not exist
fatal: not a git repository (or any of the parent directories): .git
Deploying IIS_Test_Site to inblr-uvm-1118
PLAY [IIS Update] **************************************************************TASK [create new website's directory] ******************************************ok: [inblr-uvm-1118.eu.uis.unisys.com]TASK [stop website] ************************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [unzip code to site path] *************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]TASK [start website] ***********************************************************changed: [inblr-uvm-1118.eu.uis.unisys.com]PLAY RECAP *********************************************************************inblr-uvm-1118.eu.uis.unisys.com : ok=4    changed=3    unreachable=0    failed=0   /tmp/tmpf3sh33k1
INFO: Deploying Component IIS_Test_Site to Server USTR-UVM-15937
fatal: repository '.' does not exist
fatal: not a git repository (or any of the parent directories): .git
Deploying IIS_Test_Site to USTR-UVM-15937
PLAY [IIS Update] **************************************************************TASK [create new website's directory] ******************************************ok: [USTR-UVM-15937.na.uis.unisys.com]TASK [stop website] ************************************************************ok: [USTR-UVM-15937.na.uis.unisys.com]TASK [unzip code to site path] *************************************************fatal: [USTR-UVM-15937.na.uis.unisys.com]: FAILED! => {"changed": false, "dest": "c:\\inetpub\\testsite", "msg": "File 'c:\\staging\\site.zip' does not exist.", "removed": false, "src": "c:\\staging\\site.zip"} to retry, use: --limit @/tmp/tmp44k1vhdw/runit.retryPLAY RECAP *********************************************************************USTR-UVM-15937.na.uis.unisys.com : ok=2    changed=0    unreachable=0    failed=1   /tmp/tmp44k1vhdw
INFO: Completed Parallel Deployment
INFO: Starting Sequential Deployment of Component WIN_Firewall
INFO: Deploying Component WIN_Firewall using Custom Action WIN_Firewall_Rule_Action
fatal: repository '.' does not exist
fatal: not a git repository (or any of the parent directories): .git
Deploying WIN_Firewall to inblr-uvm-1118
PLAY [Add/Update Windows Firewall Rule] ****************************************TASK [Add/Update Windows Firewall Rule] ****************************************ok: [inblr-uvm-1118.eu.uis.unisys.com]PLAY RECAP *********************************************************************inblr-uvm-1118.eu.uis.unisys.com : ok=1    changed=0    unreachable=0    failed=0   /tmp/tmp7w0dwntv
INFO: Return code is 0
INFO: Deploying Component WIN_Firewall using Custom Action WIN_Firewall_Rule_Action
fatal: repository '.' does not exist
fatal: not a git repository (or any of the parent directories): .git
Deploying WIN_Firewall to USTR-UVM-15937
PLAY [Add/Update Windows Firewall Rule] ****************************************TASK [Add/Update Windows Firewall Rule] ****************************************ok: [USTR-UVM-15937.na.uis.unisys.com]PLAY RECAP *********************************************************************USTR-UVM-15937.na.uis.unisys.com : ok=1    changed=0    unreachable=0    failed=0   /tmp/tmpt7vbuu5z
INFO: Return code is 0
INFO: Completed Parallel Deployment
INFO: Time taken 0.061199 seconds

image

CC: @svisagan83

sbtaylor15 commented 5 years ago

@piyush94 - we added code to catch the array creds["repo"] from the DeployHub Groovy and turn it into an array instead of a literal. Please pull the latest image.

CC: @svisagan83

sbtaylor15 commented 5 years ago

@piyush94 we updated the iis.yml to include additional parameters that allows for customization of the website, ie port, application pool, binding ip, ssl etc. The iis.yml will create the website if it doesn't exists.

Please pull these updates onto your fork.

image

CC: @svisagan83