search

conjurdemos / kubernetes-conjur-demo

Demo application for Conjur Kubernetes integration
11 stars 14 forks source link

Openshift 4.3 Jenkins pipeline hangs on namespace delete #125

Closed diverdane closed 3 years ago

diverdane commented 3 years ago

Summary

The Jenkins pipeline test for OpenShift 4.3 is hanging when the scripts try to clean up the application namespace following a successful deployment of Conjur + applications.

The console logs show:


[2021-01-22T15:26:09.815Z] Login Succeeded
[2021-01-22T15:26:10.751Z] NAME                                                          READY     STATUS      RESTARTS   AGE
[2021-01-22T15:26:10.751Z] secretless-mysql-0                                            1/1       Running     0          68s
[2021-01-22T15:26:10.751Z] summon-init-mysql-0                                           1/1       Running     0          68s
[2021-01-22T15:26:10.751Z] summon-sidecar-mysql-0                                        1/1       Running     0          68s
[2021-01-22T15:26:10.751Z] test-app-secretless-1-deploy                                  0/1       Completed   0          62s
[2021-01-22T15:26:10.751Z] test-app-secretless-1-qnch7                                   2/2       Running     0          57s
[2021-01-22T15:26:10.751Z] test-app-summon-init-1-4h7mc                                  1/1       Running     0          47s
[2021-01-22T15:26:10.751Z] test-app-summon-init-1-deploy                                 0/1       Completed   0          50s
[2021-01-22T15:26:10.751Z] test-app-summon-sidecar-1-deploy                              0/1       Completed   0          56s
[2021-01-22T15:26:10.751Z] test-app-summon-sidecar-1-h7rmf                               2/2       Running     0          51s
[2021-01-22T15:26:10.751Z] test-app-with-host-outside-apps-branch-summon-init-1-6d9lg    1/1       Running     0          40s
[2021-01-22T15:26:10.751Z] test-app-with-host-outside-apps-branch-summon-init-1-deploy   0/1       Completed   0          44s
[2021-01-22T15:26:11.693Z] Login successful.
[2021-01-22T15:26:11.693Z] 
[2021-01-22T15:26:11.693Z] You have access to 60 projects, the list has been suppressed. You can list all projects with 'oc projects'
[2021-01-22T15:26:11.693Z] 
[2021-01-22T15:26:11.693Z] Using project "default".
[2021-01-22T15:26:11.951Z] namespace "test-app-5-fcd9c9be-3" deleted
[2021-01-22T19:33:59.510Z] Sending interrupt signal to process
[2021-01-22T19:34:11.408Z] Sending interrupt signal to process
[2021-01-22T19:34:12.934Z] signal: terminated
[2021-01-22T19:34:12.940Z] script returned exit code 127

This does not happen for other versions of OpenShift.

This appears to be related to this OC 4.3 bug: https://bugzilla.redhat.com/show_bug.cgi?id=1798282
and can likely be fixed by deleting finalizers on Kubernetes services before deleting the corresponding namespace.

## Steps to Reproduce
Look at any CI test results on master branch.

## Expected Results
Tests pass for OC version 4.3.

## Actual Results (including error logs, if applicable)
Tests for OC version 4.3 hang after attempt is made to delete the application namespace.

## Reproducible
   * [x] Always 
   * [ ] Sometimes
   * [ ] Non-Reproducible

## Version/Tag number
Latest master.

## Environment setup
OpenShift Version 4.3

## Additional Information
diverdane commented 3 years ago

Closing this issue. This issue was fixed for this repository with #126 and #129.

Note that this issue still exists in the cyberark/kubernetes-conjur-deploy scripts, so the problem will still be seen in E2E testing that uses cyberark/kubernetes-conjur-deploy along with this repo (conjurdemos/kubernetes-conjur-demo).