donbourne
commented
4 years ago looks like it's due to 2 commands not being available on the image.
during container startup the following message appears once:
/work/start_server.sh: line 61: cmp: command not foundthen repeatedly, while the image is running:
/work/start_server.sh: line 93: ps: command not foundthe script that's emitting these messages is https://github.com/WASdev/ci.docker.websphere-traditional/blob/master/docker-build/9.0.5.2/scripts/start_server.sh the script relies on ps to see if logViewer is up, and will start a new copy if it thinks it is down...
So, I think the problem is that the newer images don't have the cmp and ps commands. Either those need to be added to the image or the start_server.sh script needs to be changed to not rely on them.
hanczaryk
I had briefly chatted with Glenn Marcy on slack who pointed me to open the issue I've observed here.
I have successfully been creating my docker image for a while using 9.0.5.2 with the line below. FROM ibmcom/websphere-traditional:9.0.5.2-ubi
I'm able to successfully build and deploy this image. Here is the output from 'oc get pods' showing that it has been up and running for over an hour with no restarts. It serves app requests at https://hanczar-b2-hanczar.apps.9.46.124.17.nip.io/B/
[root@avl-rhos1 tWAS_B]# oc get pods NAME READY STATUS RESTARTS AGE hanczar-b2-1-tgcjf 1/1 Running 0 1h
Today, I attempted to use the new 9.0.5.3 image that I saw was added to https://hub.docker.com/r/ibmcom/websphere-traditional/tags with the line below FROM ibmcom/websphere-traditional:9.0.5.3-ubi
I was able to build and deploy that image. It successfully served app requests at https://hanczar-b3-hanczar.apps.9.46.124.17.nip.io/B/ but I noticed that a large number of java processes were being started on my OCP 31 system.
After about 5 minutes, I had over 150 java processes that reference LogViewer like the one I've shown below.
This growth in java processes would eat up all the free memory as shown by top until the pod restarted, in this case it was after about 7 minutes. Once the pod restarted, the java processes were removed, the pod again served app requests until the java process growth began again.
Glenn mentioned that 9.0.5.3 uses ubi8 and that I should attempt with 9.0.5.2-ubi8 to see if it also occurred.
So, I attempted to use the 9.0.5.2-ubi8 image with the line below FROM ibmcom/websphere-traditional:9.0.5.2-ubi8
I encountered the same experience as 9.0.5.3. An increase in java processes and the pod restarted. The following 'oc get pods' shows that it restarted the pod a few times.
[root@avl-rhos1 tWAS_B]# oc get pods NAME READY STATUS RESTARTS AGE hanczar-b2-1-tgcjf 1/1 Running 0 1h hanczar-b28-1-z7cwx 1/1 Running 2 45m
Here is the top output head top - 12:38:58 up 87 days, 21:50, 1 user, load average: 9.91, 13.40, 21.87 Tasks: 847 total, 2 running, 845 sleeping, 0 stopped, 0 zombie %Cpu(s): 15.7 us, 17.7 sy, 0.3 ni, 66.0 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st KiB Mem : 16266524 total, 284260 free, 13549604 used, 2432660 buff/cache KiB Swap: 0 total, 0 free, 0 used. 2350900 avail Mem
At this point in time there were 163 java processes with 157 of them referencing LogViewer shown above.
I deleted the hanczar-28 application and observed the following.