Closed schmiuwe closed 1 year ago
callumpjackson
commented
1 year ago Hi Uwe - sorry to see that you are having difficulties. Was this during a helm upgrade or for a new deployment. Also could you provide the parameters you are using so I can reproduce locally. Thanks Callum
schmiuwe
commented
1 year ago Hi Callum,
it happened during upgrading the cluster using the new IBM MQ helm chart. I also tested purging the whole Qmgr and redeploy complete app but same issue no matter if I update or reinstall from scratch. After using 7.0.0 again it worked fine.
Thank you, Uwe
From: callumpjackson @.> Sent: Wednesday, July 19, 2023 4:01 PM To: ibm-messaging/mq-helm @.> Cc: Schmiedel Uwe, FG-232 @.>; Author @.> Subject: Re: [ibm-messaging/mq-helm] Native HA is unavailable - AMQ5708E - Error 93 for v7.0.1 (Issue #51)
Sent from outside the BMW organization - be CAUTIOUS, particularly with links and attachments.
Absender außerhalb der BMW Organisation - Bitte VORSICHT beim Öffnen von Links und Anhängen.
Hi Uwe - sorry to see that you are having difficulties. Was this during a helm upgrade or for a new deployment. Also could you provide the parameters you are using so I can reproduce locally. Thanks Callum
— Reply to this email directly, view it on GitHubhttps://github.com/ibm-messaging/mq-helm/issues/51#issuecomment-1642146584, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6UGXEMJLCWWBQLDXGMOLDDXQ7SA7ANCNFSM6AAAAAA2PZJOR4. You are receiving this because you authored the thread.Message ID: @.***>
callumpjackson
commented
1 year ago Understood - could you provide the parameters you are using so I can recreate. If they contain any sensitive data I'm happy to jump on a webex session and observe.
schmiuwe
commented
1 year ago Hi Callum,
attached the charts, one qmgr chart that references the base chart … is this all you need?
Thank you, Uwe
From: callumpjackson @.> Sent: Wednesday, July 19, 2023 4:06 PM To: ibm-messaging/mq-helm @.> Cc: Schmiedel Uwe, FG-232 @.>; Author @.> Subject: Re: [ibm-messaging/mq-helm] Native HA is unavailable - AMQ5708E - Error 93 for v7.0.1 (Issue #51)
Sent from outside the BMW organization - be CAUTIOUS, particularly with links and attachments.
Absender außerhalb der BMW Organisation - Bitte VORSICHT beim Öffnen von Links und Anhängen.
Understood - could you provide the parameters you are using so I can recreate. If they contain any sensitive data I'm happy to jump on a webex session and observe.
— Reply to this email directly, view it on GitHubhttps://github.com/ibm-messaging/mq-helm/issues/51#issuecomment-1642155544, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6UGXEI74HAKU7UT7DCNHMTXQ7SU7ANCNFSM6AAAAAA2PZJOR4. You are receiving this because you authored the thread.Message ID: @.***>
callumpjackson
commented
1 year ago Hi Uwe, I’ve good news and bad news. I tried to reproduce the issue and tried the latest MQ CD release and this works as expected and originally tested.
2023-07-19T15:00:43.097Z CPU architecture: amd64
2023-07-19T15:00:43.097Z Linux kernel version: 4.18.0-372.49.1.el8_6.x86_64
2023-07-19T15:00:43.098Z Container runtime: kube
2023-07-19T15:00:43.098Z Base image: Red Hat Enterprise Linux 8.8 (Ootpa)
2023-07-19T15:00:43.098Z Running as user ID 1000680000 with primary group 0, and supplementary groups 0,1000680000
2023-07-19T15:00:43.098Z Capabilities: none
2023-07-19T15:00:43.098Z seccomp enforcing mode: filtering
2023-07-19T15:00:43.098Z Process security attributes: system_u:system_r:container_t:s0:c15,c26
2023-07-19T15:00:43.098Z Detected 'ext4' volume mounted to /mnt/mqm
2023-07-19T15:00:43.098Z Environment variable LOG_FORMAT is deprecated. Use MQ_LOGGING_CONSOLE_FORMAT instead.
2023-07-19T15:00:43.343Z Using queue manager name: secureapphelm
2023-07-19T15:00:43.494Z Created directory structure under /var/mqm
2023-07-19T15:00:43.494Z Image created: 2023-06-05T08:54:47+00:00
2023-07-19T15:00:43.494Z Image tag: ibm-mqadvanced-server-dev:9.3.3.0-r1.20230605085013.3599852-amd64
2023-07-19T15:00:43.541Z MQ version: 9.3.3.0
2023-07-19T15:00:43.541Z MQ level: p933-L230531
2023-07-19T15:00:43.541Z MQ license: Developer
2023-07-19T15:00:46.628Z Starting web server
2023-07-19T15:00:49.622Z FIPS cryptography is not enabled.
2023-07-19T15:00:49.622Z Creating queue manager secureapphelm
2023-07-19T15:01:03.812Z Created queue manager
2023-07-19T15:01:03.913Z Removing existing ServiceComponent configuration
2023-07-19T15:00:51.610Z AMQ6287I: IBM MQ V9.3.3.0 (p933-L230531). [CommentInsert1(Linux 4.18.0-372.49.1.el8_6.x86_64 (MQ Linux (x86-64 platform) 64-bit)), CommentInsert2(/opt/mqm (Installation1)), CommentInsert3(9.3.3.0 (p933-L230531))]
2023-07-19T15:00:51.727Z AMQ7467I: The oldest log file required to start queue manager secureapphelm is S0000000.LOG. [CommentInsert1(secureapphelm), CommentInsert2(S0000000.LOG)]
2023-07-19T15:00:51.729Z AMQ7468I: The oldest log file required to perform media recovery of queue manager secureapphelm is S0000000.LOG. [CommentInsert1(secureapphelm), CommentInsert2(S0000000.LOG)]
2023-07-19T15:00:53.307Z AMQ7229I: 5 log records accessed on queue manager 'secureapphelm' during the log replay phase. [ArithInsert1(5), CommentInsert1(secureapphelm)]
2023-07-19T15:00:53.309Z AMQ7230I: Log replay for queue manager 'secureapphelm' complete. [ArithInsert1(5), CommentInsert1(secureapphelm)]This made me review your original output and I realize that you are using 9.3.0.5, the LTS instead of CD stream. When I change the container image to 9.3.0 I am able to reproduce:
2023-07-19T15:07:19.190Z CPU architecture: amd64
2023-07-19T15:07:19.191Z Linux kernel version: 4.18.0-372.49.1.el8_6.x86_64
2023-07-19T15:07:19.191Z Container runtime: kube
2023-07-19T15:07:19.191Z Base image: Red Hat Enterprise Linux 8.6 (Ootpa)
2023-07-19T15:07:19.191Z Running as user ID 1000680000 with primary group 0, and supplementary groups 0,1000680000
2023-07-19T15:07:19.191Z Capabilities: none
2023-07-19T15:07:19.191Z seccomp enforcing mode: filtering
2023-07-19T15:07:19.191Z Process security attributes: system_u:system_r:container_t:s0:c15,c26�
2023-07-19T15:07:19.191Z Detected 'ext4' volume mounted to /mnt/mqm
2023-07-19T15:07:19.346Z Using queue manager name: secureapphelm
2023-07-19T15:07:19.420Z Created directory structure under /var/mqm
2023-07-19T15:07:19.420Z Image created: 2022-06-15T13:12:21+00:00
2023-07-19T15:07:19.420Z Image tag: ibm-mqadvanced-server-dev:9.3.0.0-r1.20220615130653.3111d48-amd64
2023-07-19T15:07:19.517Z MQ version: 9.3.0.0
2023-07-19T15:07:19.518Z MQ level: p930-L220606
2023-07-19T15:07:19.518Z MQ license: Developer
2023-07-19T15:07:23.026Z Starting web server
2023-07-19T15:07:26.207Z Creating queue manager secureapphelm
2023-07-19T15:07:26.507Z Error 93 creating queue manager: AMQ5708E: Native HA is unavailable.
2023-07-19T15:07:26.507Z /opt/mqm/bin/crtmqm: exit status 93The reason for the difference is that we made a licensing change in 9.3.2 where Native HA is available within an MQ Advanced license. This change involved the removal of an environment variable called AMQ_CLOUD_PAK. The reason why we updated between 7.0.0 and 7.0.1 is that we considered it a bug, as it should have been within Helm Charts supporting 9.3.2 onwards. The general strategy is that a single Helm Chart release corresponds to an IBM MQ release. Its fair to say that we have seen good backward and forward compatibility, so this hasn’t been a major issue to date.
In the case of the LTS this corresponds to MQ Helm Chart 3.0.0. Unless you need new features of the Helm Chart in the later releases, I would suggest it would be easier to remain on an earlier version than 7.0.1.
schmiuwe
commented
1 year ago Hi Callum,
I understand, ok fine for me as long as I know it. So we will stay with 7.0.0 for now.
Thank you, Uwe
From: callumpjackson @.> Sent: Wednesday, July 19, 2023 6:32 PM To: ibm-messaging/mq-helm @.> Cc: Schmiedel Uwe, FG-232 @.>; Author @.> Subject: Re: [ibm-messaging/mq-helm] Native HA is unavailable - AMQ5708E - Error 93 for v7.0.1 (Issue #51)
Sent from outside the BMW organization - be CAUTIOUS, particularly with links and attachments.
Absender außerhalb der BMW Organisation - Bitte VORSICHT beim Öffnen von Links und Anhängen.
Hi Uwe, I’ve good news and bad news. I tried to reproduce the issue and tried the latest MQ CD release and this works as expected and originally tested.
2023-07-19T15:00:43.097Z CPU architecture: amd64
2023-07-19T15:00:43.097Z Linux kernel version: 4.18.0-372.49.1.el8_6.x86_64
2023-07-19T15:00:43.098Z Container runtime: kube
2023-07-19T15:00:43.098Z Base image: Red Hat Enterprise Linux 8.8 (Ootpa)
2023-07-19T15:00:43.098Z Running as user ID 1000680000 with primary group 0, and supplementary groups 0,1000680000
2023-07-19T15:00:43.098Z Capabilities: none
2023-07-19T15:00:43.098Z seccomp enforcing mode: filtering
2023-07-19T15:00:43.098Z Process security attributes: system_u:system_r:container_t:s0:c15,c26
2023-07-19T15:00:43.098Z Detected 'ext4' volume mounted to /mnt/mqm
2023-07-19T15:00:43.098Z Environment variable LOG_FORMAT is deprecated. Use MQ_LOGGING_CONSOLE_FORMAT instead.
2023-07-19T15:00:43.343Z Using queue manager name: secureapphelm
2023-07-19T15:00:43.494Z Created directory structure under /var/mqm
2023-07-19T15:00:43.494Z Image created: 2023-06-05T08:54:47+00:00
2023-07-19T15:00:43.494Z Image tag: ibm-mqadvanced-server-dev:9.3.3.0-r1.20230605085013.3599852-amd64
2023-07-19T15:00:43.541Z MQ version: 9.3.3.0
2023-07-19T15:00:43.541Z MQ level: p933-L230531
2023-07-19T15:00:43.541Z MQ license: Developer
2023-07-19T15:00:46.628Z Starting web server
2023-07-19T15:00:49.622Z FIPS cryptography is not enabled.
2023-07-19T15:00:49.622Z Creating queue manager secureapphelm
2023-07-19T15:01:03.812Z Created queue manager
2023-07-19T15:01:03.913Z Removing existing ServiceComponent configuration
2023-07-19T15:00:51.610Z AMQ6287I: IBM MQ V9.3.3.0 (p933-L230531). [CommentInsert1(Linux 4.18.0-372.49.1.el8_6.x86_64 (MQ Linux (x86-64 platform) 64-bit)), CommentInsert2(/opt/mqm (Installation1)), CommentInsert3(9.3.3.0 (p933-L230531))]
2023-07-19T15:00:51.727Z AMQ7467I: The oldest log file required to start queue manager secureapphelm is S0000000.LOG. [CommentInsert1(secureapphelm), CommentInsert2(S0000000.LOG)]
2023-07-19T15:00:51.729Z AMQ7468I: The oldest log file required to perform media recovery of queue manager secureapphelm is S0000000.LOG. [CommentInsert1(secureapphelm), CommentInsert2(S0000000.LOG)]
2023-07-19T15:00:53.307Z AMQ7229I: 5 log records accessed on queue manager 'secureapphelm' during the log replay phase. [ArithInsert1(5), CommentInsert1(secureapphelm)]
2023-07-19T15:00:53.309Z AMQ7230I: Log replay for queue manager 'secureapphelm' complete. [ArithInsert1(5), CommentInsert1(secureapphelm)]
This made me review your original output and I realize that you are using 9.3.0.5, the LTS instead of CD stream. When I change the container image to 9.3.0 I am able to reproduce:
2023-07-19T15:07:19.190Z CPU architecture: amd64
2023-07-19T15:07:19.191Z Linux kernel version: 4.18.0-372.49.1.el8_6.x86_64
2023-07-19T15:07:19.191Z Container runtime: kube
2023-07-19T15:07:19.191Z Base image: Red Hat Enterprise Linux 8.6 (Ootpa)
2023-07-19T15:07:19.191Z Running as user ID 1000680000 with primary group 0, and supplementary groups 0,1000680000
2023-07-19T15:07:19.191Z Capabilities: none
2023-07-19T15:07:19.191Z seccomp enforcing mode: filtering
2023-07-19T15:07:19.191Z Process security attributes: system_u:system_r:container_t:s0:c15,c26�
2023-07-19T15:07:19.191Z Detected 'ext4' volume mounted to /mnt/mqm
2023-07-19T15:07:19.346Z Using queue manager name: secureapphelm
2023-07-19T15:07:19.420Z Created directory structure under /var/mqm
2023-07-19T15:07:19.420Z Image created: 2022-06-15T13:12:21+00:00
2023-07-19T15:07:19.420Z Image tag: ibm-mqadvanced-server-dev:9.3.0.0-r1.20220615130653.3111d48-amd64
2023-07-19T15:07:19.517Z MQ version: 9.3.0.0
2023-07-19T15:07:19.518Z MQ level: p930-L220606
2023-07-19T15:07:19.518Z MQ license: Developer
2023-07-19T15:07:23.026Z Starting web server
2023-07-19T15:07:26.207Z Creating queue manager secureapphelm
2023-07-19T15:07:26.507Z Error 93 creating queue manager: AMQ5708E: Native HA is unavailable.
2023-07-19T15:07:26.507Z /opt/mqm/bin/crtmqm: exit status 93
The reason for the difference is that we made a licensing change in 9.3.2 where Native HA is available within an MQ Advanced license. This change involved the removal of an environment variable called AMQ_CLOUD_PAK. The reason why we updated between 7.0.0 and 7.0.1 is that we considered it a bug, as it should have been within Helm Charts supporting 9.3.2 onwards. The general strategy is that a single Helm Chart release corresponds to an IBM MQ release. Its fair to say that we have seen good backward and forward compatibility, so this hasn’t been a major issue to date.
In the case of the LTS this corresponds to MQ Helm Chart 3.0.0. Unless you need new features of the Helm Chart in the later releases, I would suggest it would be easier to remain on an earlier version than 7.0.1.
— Reply to this email directly, view it on GitHubhttps://github.com/ibm-messaging/mq-helm/issues/51#issuecomment-1642403475, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A6UGXEPOWYYDRVPXID5EEU3XRADWXANCNFSM6AAAAAA2PZJOR4. You are receiving this because you authored the thread.Message ID: @.***>
callumpjackson
commented
1 year ago Thanks - closing issue.
Dear all,
using the helm chart v7.0.1 we are getting the following error:
2023-07-19T10:47:17.950Z Using queue manager name: P01M01G 2023-07-19T10:47:17.950Z CPU architecture: amd64 2023-07-19T10:47:17.950Z Linux kernel version: 5.15.0-1035-azure 2023-07-19T10:47:17.950Z Base image: Red Hat Enterprise Linux 8.8 (Ootpa) 2023-07-19T10:47:17.950Z Running as user ID 1001 with primary group 0, and supplementary groups 0,2000 2023-07-19T10:47:17.950Z Capabilities: none 2023-07-19T10:47:17.950Z seccomp enforcing mode: disabled 2023-07-19T10:47:17.950Z Process security attributes: cri-containerd.apparmor.d (enforce) 2023-07-19T10:47:17.951Z Detected 'ext4' volume mounted to /mnt/mqm-data 2023-07-19T10:47:17.951Z Detected 'ext4' volume mounted to /mnt/mqm-log 2023-07-19T10:47:17.951Z Detected 'ext4' volume mounted to /mnt/mqm 2023-07-19T10:47:17.958Z Created directory structure under /var/mqm 2023-07-19T10:47:17.958Z Image created: 2023-05-25T14:16:50+00:00 2023-07-19T10:47:17.958Z Image tag: ibm-mqadvanced-server:9.3.0.5-r3.20230525140420.7d14e60-amd64 2023-07-19T10:47:17.977Z MQ version: 9.3.0.5 2023-07-19T10:47:17.977Z MQ level: p930-005-230413 2023-07-19T10:47:17.977Z MQ license: Production 2023-07-19T10:47:23.215Z Creating queue manager P01M01G 2023-07-19T10:47:23.241Z Error 93 creating queue manager: AMQ5708E: Native HA is unavailable.
2023-07-19T10:47:23.241Z /opt/mqm/bin/crtmqm: exit status 93
I just reverted back to v7.0.0. With this version the pods do come up.
Thank you, Uwe