search

zowe / zowe-install-packaging

Packaging repository for the Zowe install scripts and files
Eclipse Public License 2.0
27 stars 51 forks source link

Handle IPCS queue full gracefully #3444

Open pablocarle opened 1 year ago

pablocarle commented 1 year ago

Describe the bug

Possibly related to #326 Related to #3443 We've been receiving reports, and we are able to reproduce in our systems. When IPCS queues get filled up, Zowe fails to prepare the instance and does not start.

https://www.ibm.com/docs/en/sdk-nodejs-zos/14.0?topic=zos-troubleshooting#d3724e264

This topic does not seem to be covered in the public documentation, either as troubleshooting or properly sizing the queues.

To Reproduce Steps to reproduce the behavior:

  1. Either make sure to stop Zowe incorrectly so the IPC queue will not clean up or reduce the size.
  2. Start Zowe
  3. Depending on the version used, logs could appear of the error or not. The error and solution is described in the IBM link above.

Expected behavior Stop execution with appropriate message in logs.

1000TurquoisePogs commented 1 year ago

I think we do cleanup of the queue at "internal start prepare" via this script https://github.com/search?q=repo%3Azowe%2Fzowe-install-packaging%20cleanup-ipc-mq&type=code but the problem is that this script is not called at "start component" time, so, if a component crashes post-startup, the launcher restarts it, skipping the "prepare" step where this cleanup gets run. So, we could call this script at the start phase too, so that continued restarts does not exhaust the finite queue resources.