New runner registration is never reattempted on failure

[akomakom]

If Github_actions_runner::Instance[x]/Exec[x-run_configure_install_runner.sh fails, the puppet run will correctly skip the next steps (starting the service). However, on the next puppet run, it will not attempt to run configure again and will instead try to start the service (and "succeed", each time).

As a workaround, deleting the install dir ($base_dir_name) will cause the class to work correctly next time.

Note: configure may fail due to a transient problem (eg network issue, github enterprise misconfiguration, auth misconfiguration, etc).

ENV

Puppet: 5.5 telefonica-github_actions_runner: 0.40 OS: Ubuntu 18.04 and 20.04

Code

    class { 'github_actions_runner':                                                                                                                                                            
      user                  => $user,                                                                                                                                                           
      group                 => $user,                                                                                                                                                           
      base_dir_name         => $install_path,                                                                                                                                                   
      github_domain         => 'https://github.mycompany.com',                                                                                                                                 
      github_api            => 'https://github.mycompany.com/api/v3',                                                                                                                          
      personal_access_token => $pac,                                                                                                                                                            
      instances             => {                                                                                                                                                                
        'infra' => {                                                                                                                                                                            
          org_name => 'Infra',                                                                                                                                                                
        },                                                                                                                                                                                      
        'legacy' => {                                                                                                                                                                           
          org_name => 'Legacy',                                                                                                                                                            
        }                                                                                                                                                                                       

      }                                                                                                                                                                                         
    }                                                                                                                                                                                                  

I haven't looked at the code yet, but I'm happy to make a PR if I figure it out (pointers welcome). Reporting this for now to keep track.

[mrodm]

Sorry for the late response, and thanks for reporting this issue.

The problem is that the configure exec resource is just triggered (thanks to refreshonly) when the file resource is created or updated in (manifests/instance.pp).

Here it is a link to the resources: https://github.com/Telefonica/puppet-github-actions-runner/blob/master/manifests/instance.pp#L117-L149

Regarding misconfigurations, I would say puppet would try to run again configure script, since the script content would be updated with the new values. At least when changing the personal_access_token value. However, regarding network issues during configuration script execution, that is more problematic.

For the latter scenario, a solution that could work could be add a new exec resource checking for the presence of a file that is created once the configuration script is executed:

• I've checked that the configuration script creates a new shellscript runsvc.sh inside the folder of the new runner as last step:
  • https://github.com/Telefonica/puppet-github-actions-runner/blob/master/templates/configure_install_runner.sh.epp#L39
• A new exec resource checking for the presence of that file (runsvc.sh) would be needed.
  • This new exec would notify to the already existing exec resource (the one configuring the runner).
• With the above, the exec executing the config script would be executed in these two scenarios:
  • config contents script changes
  • runsvc.sh does not exist

To be tested yet, but in this pull request you can find the code that it could solve this issue: https://github.com/Telefonica/puppet-github-actions-runner/pull/24

diff --git manifests/instance.pp manifests/instance.pp
index b4d50b9..f82142e 100644
--- manifests/instance.pp
+++ manifests/instance.pp
@@ -132,6 +132,14 @@ define github_actions_runner::instance (
     require => Archive["${instance_name}-${archive_name}"],
   }

+  exec { "${instance_name}-check-runner-configured":
+    user        => $user,
+    cwd         => "${github_actions_runner::root_dir}/${instance_name}",
+    command     => "true",
+    unless      => "test -f ${github_actions_runner::root_dir}/${instance_name}/runsvc.sh",
+    path        => ["/bin", "/usr/bin"],
+    notify      => Exec["${instance_name}-run_configure_install_runner.sh]",
+  }
+
   exec { "${instance_name}-ownership":
     user        => $user,
     cwd         => $github_actions_runner::root_dir,

It's still missing some tests, we'll work on that. In any case, you can try to use that branch/PR in your puppet to check whether or not that PR fixes this issue :crossed_fingers: .

[mrodm]

I've merged pull request #24 that should resolve this issue. These changes will be available in the next release.

@akomakom If you find any other issue in this regard, please re-open this issue.

Thanks!