Zowe fails to start with InternalError: invalid opcode (op=0 .invalid., pc=0)

[lucasortega99]

Describe the bug Zowe fails to start with InternalError: invalid opcode (op=0 .invalid., pc=0) showing on the log Issuing the command 'zwe internal get-launch-components --config "FILE(/global/zowe/zowe.yaml)"' manually i get the following error: InternalError: ss_check bytecode buffer overflow (op=232 [if_false8], pc=7). Function=

changing configmgr to false solves this issue, but start up time takes really long.

To Reproduce Steps to reproduce the behavior:

1. Start Zowe

Expected behavior Zowe starts

Logs

SYSLOG output

> MapStart zowe: MapStart setup: MapStart dataset: MapStart prefix: Scalar: (len=10)SYS1X.ZOWE proclib: Scalar: (len=12)USER.PROCLIB parmlib: Scalar: (len=23)SYS1X.ZOWE.CUST.PARMLIB parmlibMembers: MapStart zis: Scalar: (len=8)ZWESIP00 MapEnd jcllib: Scalar: (len=22)SYS1X.ZOWE.CUST.JCLLIB loadlib: Scalar: (len=19)SYS1X.ZOWE.SZWELOAD authLoadlib: Scalar: (len=19)SYS1X.ZOWE.SZWEAUTH authPluginLib: Scalar: (len=23)SYS1X.ZOWE.CUST.ZWESAPL MapEnd security: MapStart product: Scalar: (len=4)RACF groups: MapStart admin: Scalar: (len=8)ZWEADMIN stc: Scalar: (len=8)ZWEADMIN sysProg: Scalar: (len=8)ZWEADMIN MapEnd users: MapStart zowe: Scalar: (len=8)ZWESVUSR zis: Scalar: (len=8)ZWESIUSR MapEnd stcs: MapStart zowe: Scalar: (len=8)ZWESLSTC zis: Scalar: (len=8)ZWESISTC aux: Scalar: (len=8)ZWESASTC MapEnd MapEnd vsam: MapStart mode: Scalar: (len=6)NONRLS volume: Scalar: (len=0) storageClass: Scalar: (len=0) MapEnd MapEnd runtimeDirectory: Scalar: (len=13)/usr/lpp/zowe logDirectory: Scalar: (len=17)/global/zowe/logs workspaceDirectory: Scalar: (len=22)/global/zowe/workspace extensionDirectory: Scalar: (len=23)/global/zowe/extensions useConfigmgr: Scalar: (len=4)true configmgr: MapStart validation: Scalar: (len=16)COMPONENT-COMPAT MapEnd job: MapStart name: Scalar: (len=6)ZWE1SV prefix: Scalar: (len=4)ZWE1 MapEnd rbacProfileIdentifier: Scalar: (len=1)1 cookieIdentifier: Scalar: (len=1)1 externalDomains: SequenceStart (block) Scalar: (len=19) SequenceEnd externalPort: Scalar: (len=4)7554 launchScript: MapStart logLevel: Scalar: (len=5)debug onComponentConfigureFail: Scalar: (len=4)warn MapEnd certificate: MapStart keystore: MapStart type: Scalar: (len=9)JCERACFKS file: Scalar: (len=33)safkeyring://ZWESVUSR/ZoweKeyring password: Scalar: (len=8)password alias: Scalar: (len=16)Zowe Certificate MapEnd truststore: MapStart type: Scalar: (len=9)JCERACFKS file: Scalar: (len=33)safkeyring://ZWESVUSR/ZoweKeyring password: Scalar: (len=8)password MapEnd pem: MapStart key: Scalar: (len=0) certificate: Scalar: (len=0) certificateAuthorities: Scalar: (len=45)safkeyring:////ZWESVUSR/ZoweKeyring&COMODO-CA MapEnd MapEnd verifyCertificates: Scalar: (len=6)STRICT MapEnd java: MapStart home: Scalar: (len=21)/usr/lpp/java/current MapEnd node: MapStart home: Scalar: (len=14)/global/nodejs MapEnd zOSMF: MapStart host: Scalar: (len=19) port: Scalar: (len=3)443 applId: Scalar: (len=7)IZUDFLT MapEnd components: MapStart gateway: MapStart enabled: Scalar: (len=4)true port: Scalar: (len=4)7554 debug: Scalar: (len=5)false apiml: MapStart security: MapStart auth: MapStart provider: Scalar: (len=5)zosmf zosmf: MapStart jwtAutoconfiguration: Scalar: (len=4)auto serviceId: Scalar: (len=5)zosmf MapEnd MapEnd authorization: MapStart endpoint: MapStart enabled: Scalar: (len=5)false MapEnd provider: Scalar: (len=0) MapEnd x509: MapStart enabled: Scalar: (len=5)false MapEnd MapEnd MapEnd server: MapStart internal: MapStart enabled: Scalar: (len=5)false port: Scalar: (len=4)7550 ssl: MapStart enabled: Scalar: (len=5)false MapEnd MapEnd MapEnd MapEnd metrics-service: MapStart enabled: Scalar: (len=5)false port: Scalar: (len=4)7551 debug: Scalar: (len=5)false MapEnd cloud-gateway: MapStart enabled: Scalar: (len=5)false port: Scalar: (len=4)7563 debug: Scalar: (len=5)false MapEnd api-catalog: MapStart enabled: Scalar: (len=4)true port: Scalar: (len=4)7552 debug: Scalar: (len=5)false MapEnd discovery: MapStart enabled: Scalar: (len=4)true port: Scalar: (len=4)7553 debug: Scalar: (len=5)false MapEnd caching-service: MapStart enabled: Scalar: (len=5)false port: Scalar: (len=4)7555 debug: Scalar: (len=5)false storage: MapStart evictionStrategy: Scalar: (len=6)reject mode: Scalar: (len=4)VSAM size: Scalar: (len=5)10000 vsam: MapStart name: Scalar: (len=0) MapEnd infinispan: MapStart jgroups: MapStart port: Scalar: (len=4)7600 MapEnd MapEnd MapEnd MapEnd app-server: MapStart enabled: Scalar: (len=4)true port: Scalar: (len=4)7556 MapEnd zss: MapStart enabled: Scalar: (len=4)true port: Scalar: (len=4)7557 crossMemoryServerName: Scalar: (len=10)ZWESIS_STD tls: Scalar: (len=4)true agent: MapStart jwt: MapStart fallback: Scalar: (len=4)true MapEnd MapEnd MapEnd jobs-api: MapStart enabled: Scalar: (len=4)true debug: Scalar: (len=5)false port: Scalar: (len=4)7558 MapEnd files-api: MapStart enabled: Scalar: (len=4)true debug: Scalar: (len=5)false port: Scalar: (len=4)7559 MapEnd explorer-jes: MapStart enabled: Scalar: (len=4)true MapEnd explorer-mvs: MapStart enabled: Scalar: (len=4)true MapEnd explorer-uss: MapStart enabled: Scalar: (len=4)true MapEnd MapEnd MapEnd config before template eval: config after template eval: vdref object i=0 k=zowe vdref object i=1 k=runtimeDirectory 2023-06-30 19:46:32 ZWESVUSR INFO ZWEL0017I ROOT_DIR is '/usr/lpp/zowe' about to build schema for '/usr/lpp/zowe/schemas/zowe-yaml-schema.json' after setjmp normal JSONSchema build JSONSchema build /$defs/port JSONSchema build /$defs/certificate *** INFO *** untyped value at path: /$defs/certificate JSONSchema build /$defs/certificate/oneOf/0 *** INFO *** untyped value at path: /$defs/certificate/oneOf/0 JSONSchema build /$defs/certificate/oneOf/1 *** INFO *** untyped value at path: /$defs/certificate/oneOf/1 JSONSchema build /$defs/pkcs12-certificate JSONSchema build /$defs/pkcs12-certificate/keystore JSONSchema build /$defs/pkcs12-certificate/keystore/type JSONSchema build /$defs/pkcs12-certificate/keystore/file JSONSchema build /$defs/pkcs12-certificate/keystore/password JSONSchema build /$defs/pkcs12-certificate/keystore/alias JSONSchema build /$defs/pkcs12-certificate/truststore JSONSchema build /$defs/pkcs12-certificate/truststore/type JSONSchema build /$defs/pkcs12-certificate/truststore/file JSONSchema build /$defs/pkcs12-certificate/truststore/password JSONSchema build /$defs/pkcs12-certificate/pem JSONSchema build /$defs/pkcs12-certificate/pem/key JSONSchema build /$defs/pkcs12-certificate/pem/certificate JSONSchema build $defs/pkcs12-certificate/pem/certificateAuthorities *** INFO *** untyped value at path: /$defs/pkcs12-certificate/pem/certificateAuthorities JSONSchema build /$defs/pkcs12-certificate/pem/certificateAuthorities/oneOf/0 JSONSchema build /$defs/pkcs12-certificate/pem/certificateAuthorities/oneOf/1 JSONSchema build /$defs/pkcs12-certificate/pem/certificateAuthorities/oneOf/1/items JSONSchema build /$defs/keyring-certificate JSONSchema build /$defs/keyring-certificate/keystore JSONSchema build /$defs/keyring-certificate/keystore/type JSONSchema build /$defs/keyring-certificate/keystore/file JSONSchema build /$defs/keyring-certificate/keystore/password JSONSchema build /$defs/keyring-certificate/keystore/alias JSONSchema build /$defs/keyring-certificate/truststore JSONSchema build /$defs/keyring-certificate/truststore/type JSONSchema build /$defs/keyring-certificate/truststore/file JSONSchema build /$defs/keyring-certificate/truststore/password JSONSchema build /$defs/keyring-certificate/pem JSONSchema build /$defs/keyring-certificate/pem/key JSONSchema build /$defs/keyring-certificate/pem/certificate JSONSchema build /$defs/keyring-certificate/pem/certificateAuthorities *** INFO *** untyped value at path: /$defs/keyring-certificate/pem/certificateAuthorities JSONSchema build /$defs/keyring-certificate/pem/certificateAuthorities/oneOf/0 JSONSchema build /$defs/keyring-certificate/pem/certificateAuthorities/oneOf/1 JSONSchema build /$defs/keyring-certificate/pem/certificateAuthorities/oneOf/1/items JSONSchema build /$defs/component JSONSchema build /$defs/component/enabled JSONSchema build /$defs/component/certificate *** INFO *** untyped value at path: /$defs/component/certificate JSONSchema build /$defs/component/launcher JSONSchema build /$defs/component/launcher/restartIntervals JSONSchema build /$defs/component/launcher/restartIntervals/items JSONSchema build /$defs/component/launcher/minUptime JSONSchema build /$defs/component/launcher/shareAs JSONSchema build /$defs/registryHandler JSONSchema build /$defs/registryHandler/registry JSONSchema build /$defs/registryHandler/path JSONSchema build /zowe JSONSchema build /zowe/setup JSONSchema build /zowe/setup/dataset JSONSchema build /zowe/setup/dataset/prefix JSONSchema build /zowe/setup/dataset/proclib JSONSchema build /zowe/setup/dataset/parmlib JSONSchema build /zowe/setup/dataset/parmlibMembers JSONSchema build /zowe/setup/dataset/parmlibMembers/zis *** INFO *** untyped value at path: /zowe/setup/dataset/parmlibMembers/zis JSONSchema build /zowe/setup/dataset/jcllib JSONSchema build /zowe/setup/dataset/loadlib JSONSchema build /zowe/setup/dataset/authLoadlib JSONSchema build /zowe/setup/dataset/authPluginLib JSONSchema build /zowe/setup/zis JSONSchema build /zowe/setup/zis/parmlib JSONSchema build /zowe/setup/zis/parmlib/keys JSONSchema build /zowe/setup/security JSONSchema build /zowe/setup/security/product JSONSchema build /zowe/setup/security/groups JSONSchema build /zowe/setup/security/groups/admin JSONSchema build /zowe/setup/security/groups/stc JSONSchema build /zowe/setup/security/groups/sysProg JSONSchema build /zowe/setup/security/users JSONSchema build /zowe/setup/security/users/zowe JSONSchema build /zowe/setup/security/users/zis JSONSchema build /zowe/setup/security/stcs JSONSchema build /zowe/setup/security/stcs/zowe JSONSchema build /zowe/setup/security/stcs/zis JSONSchema build /zowe/setup/security/stcs/aux JSONSchema build /zowe/setup/certificate JSONSchema build /zowe/setup/certificate/type JSONSchema build /zowe/setup/certificate/pkcs12 JSONSchema build /zowe/setup/certificate/pkcs12/directory JSONSchema build /zowe/setup/certificate/pkcs12/name JSONSchema build /zowe/setup/certificate/pkcs12/password JSONSchema build /zowe/setup/certificate/pkcs12/caAlias JSONSchema build /zowe/setup/certificate/pkcs12/caPassword JSONSchema build /zowe/setup/certificate/pkcs12/lock JSONSchema build /zowe/setup/certificate/pkcs12/import JSONSchema build /zowe/setup/certificate/pkcs12/import/keystore JSONSchema build /zowe/setup/certificate/pkcs12/import/password JSONSchema build /zowe/setup/certificate/pkcs12/import/alias JSONSchema build /zowe/setup/certificate/keyring JSONSchema build /zowe/setup/certificate/keyring/owner JSONSchema build /zowe/setup/certificate/keyring/name JSONSchema build /zowe/setup/certificate/keyring/label JSONSchema build /zowe/setup/certificate/keyring/caLabel JSONSchema build /zowe/setup/certificate/keyring/connect JSONSchema build /zowe/setup/certificate/keyring/connect/user JSONSchema build /zowe/setup/certificate/keyring/connect/label JSONSchema build /zowe/setup/certificate/keyring/import JSONSchema build /zowe/setup/certificate/keyring/import/dsName JSONSchema build /zowe/setup/certificate/keyring/import/password JSONSchema build /zowe/setup/certificate/keyring/zOSMF JSONSchema build /zowe/setup/certificate/keyring/zOSMF/ca JSONSchema build /zowe/setup/certificate/keyring/zOSMF/user JSONSchema build /zowe/setup/certificate/dname JSONSchema build /zowe/setup/certificate/dname/caCommonName JSONSchema build /zowe/setup/certificate/dname/commonName JSONSchema build /zowe/setup/certificate/dname/orgUnit JSONSchema build /zowe/setup/certificate/dname/org JSONSchema build /zowe/setup/certificate/dname/locality JSONSchema build /zowe/setup/certificate/dname/state JSONSchema build /zowe/setup/certificate/dname/country JSONSchema build /zowe/setup/certificate/validity JSONSchema build /zowe/setup/certificate/san JSONSchema build /zowe/setup/certificate/san/items JSONSchema build /zowe/setup/certificate/importCertificateAuthorities JSONSchema build /zowe/setup/certificate/importCertificateAuthorities/items JSONSchema build /zowe/setup/vsam JSONSchema build /zowe/setup/vsam/mode JSONSchema build /zowe/setup/vsam/volume JSONSchema build /zowe/setup/vsam/storageClass JSONSchema build /zowe/runtimeDirectory JSONSchema build /zowe/logDirectory JSONSchema build /zowe/workspaceDirectory JSONSchema build /zowe/extensionDirectory JSONSchema build /zowe/job JSONSchema build /zowe/job/name JSONSchema build /zowe/job/prefix JSONSchema build /zowe/network JSONSchema build /zowe/network/vipaIp JSONSchema build /zowe/network/validatePortFree JSONSchema build /zowe/extensionRegistry JSONSchema build /zowe/extensionRegistry/defaultHandler JSONSchema build /zowe/extensionRegistry/handlers JSONSchema build /zowe/extensionRegistry/handlers/patternProperties/^.*$ *** INFO *** untyped value at path: /zowe/extensionRegistry/handlers/patternProperties/^.*$ JSONSchema build /zowe/launcher JSONSchema build /zowe/launcher/restartIntervals JSONSchema build /zowe/launcher/restartIntervals/items JSONSchema build /zowe/launcher/minUptime JSONSchema build /zowe/launcher/shareAs JSONSchema build /zowe/rbacProfileIdentifier JSONSchema build /zowe/cookieIdentifier JSONSchema build /zowe/externalDomains JSONSchema build /zowe/externalDomains/items JSONSchema build /zowe/externalPort *** INFO *** untyped value at path: /zowe/externalPort JSONSchema build /zowe/environments JSONSchema build /zowe/launchScript JSONSchema build /zowe/launchScript/logLevel JSONSchema build /zowe/launchScript/onComponentConfigureFail JSONSchema build /zowe/certificate *** INFO *** untyped value at path: /zowe/certificate JSONSchema build /zowe/verifyCertificates JSONSchema build /zowe/useConfigmgr JSONSchema build /zowe/configmgr JSONSchema build /zowe/configmgr/validation JSONSchema build /java JSONSchema build /java/home JSONSchema build /node JSONSchema build /node/home JSONSchema build /zOSMF JSONSchema build /zOSMF/host JSONSchema build /zOSMF/port *** INFO *** untyped value at path: /zOSMF/port JSONSchema build /zOSMF/applId JSONSchema build /components JSONSchema build /components/patternProperties/^.*$ *** INFO *** untyped value at path: /components/patternProperties/^.*$ JSONSchema build /haInstances JSONSchema build /haInstances/patternProperties/^.*$ JSONSchema build /haInstances/patternProperties/^.*$/hostname JSONSchema build /haInstances/patternProperties/^.*$/sysname JSONSchema build /haInstances/patternProperties/^.*$/components JSONSchema build /haInstances/patternProperties/^.*$/components/patternProperties/^.*$ *** INFO *** untyped value at path: /haInstances/patternProperties/^.*$/components/patternProperties/^.*$ about to build schema for '/usr/lpp/zowe/schemas/server-common.json' after setjmp normal JSONSchema build *** INFO *** untyped value at path: JSONSchema build /$defs/semverVersion JSONSchema build /$defs/semverRange JSONSchema build /$defs/dataset JSONSchema build /$defs/datasetMember JSONSchema build /$defs/jobname JSONSchema build /$defs/user JSONSchema build /$defs/token JSONSchema build /$defs/path JSONSchema build /$defs/file JSONSchema build /$defs/reverseDomainNotation JSONSchema build /$defs/ipv4 JSONSchema build /$defs/tcpPort JSONSchema build /$defs/reservedTcpPort /zowe /zowe/setup /zowe/setup/dataset /zowe/setup/dataset/parmlibMembers /zowe/setup/security /zowe/setup/security/groups /zowe/setup/security/users /zowe/setup/security/stcs /zowe/setup/vsam /zowe/configmgr /zowe/job /zowe/launchScript /zowe/certificate /zowe/certificate/keystore /zowe/certificate/truststore /zowe/certificate/pem /zowe/certificate /zowe/certificate/keystore /zowe/certificate/truststore /zowe/certificate/pem /zowe/certificate /java /node /zOSMF /components /components/gateway /components/metrics-service /components/cloud-gateway /components/api-catalog /components/discovery /components/caching-service /components/app-server /components/zss /components/jobs-api /components/files-api /components/explorer-jes /components/explorer-mvs /components/explorer-uss 2023-06-30 19:46:32 ZWESVUSR INFO ZWEL0069I Configuration is valid vdref object i=0 k=zowe vdref object i=1 k=environments mkey='environments', pkey='setup' mkey='environments', pkey='runtimeDirectory' mkey='environments', pkey='logDirectory' mkey='environments', pkey='workspaceDirectory' mkey='environments', pkey='extensionDirectory' mkey='environments', pkey='useConfigmgr' mkey='environments', pkey='configmgr' mkey='environments', pkey='job' mkey='environments', pkey='rbacProfileIdentifier' mkey='environments', pkey='cookieIdentifier' mkey='environments', pkey='externalDomains' mkey='environments', pkey='externalPort' mkey='environments', pkey='launchScript' mkey='environments', pkey='certificate' mkey='environments', pkey='verifyCertificates' loud key='environments' ret=0x0 loud prop=0x0 Prop = 0x5008883A7C k=0x5008883A54 v=0x5008883A5C key='setup' -> 0x5008883A5C (type=4) Prop = 0x50088841BC k=0x5008884184 v=0x500888419C key='runtimeDirectory' -> 0x500888419C (type=1) Prop = 0x500888420C k=0x50088841D4 v=0x50088841E4 key='logDirectory' -> 0x50088841E4 (type=1) Prop = 0x5008884264 k=0x5008884224 v=0x500888423C key='workspaceDirectory' -> 0x500888423C (type=1) Prop = 0x50088842BC k=0x500888427C v=0x5008884294 key='extensionDirectory' -> 0x5008884294 (type=1) Prop = 0x50088842F4 k=0x50088842D4 v=0x50088842E4 key='useConfigmgr' -> 0x50088842E4 (type=2) Prop = 0x500888433C k=0x500888430C v=0x500888431C key='configmgr' -> 0x500888431C (type=4) Prop = 0x50088843CC k=0x50088843A4 v=0x50088843AC key='job' -> 0x50088843AC (type=4) Prop = 0x5008884484 k=0x5008884454 v=0x500888446C key='rbacProfileIdentifier' -> 0x500888446C (type=1) Prop = 0x50088844CC k=0x500888449C v=0x50088844B4 key='cookieIdentifier' -> 0x50088844B4 (type=1) Prop = 0x5008884554 k=0x50088844E4 v=0x50088844F4 key='externalDomains' -> 0x50088844F4 (type=5) Prop = 0x50088845B4 k=0x5008884594 v=0x50088845A4 key='externalPort' -> 0x50088845A4 (type=6) Prop = 0x50088845FC k=0x50088845CC v=0x50088845DC key='launchScript' -> 0x50088845DC (type=4) Prop = 0x50088846D4 k=0x50088846A4 v=0x50088846B4 key='certificate' -> 0x50088846B4 (type=4) Prop = 0x5008884ADC k=0x5008884AAC v=0x5008884AC4 key='verifyCertificates' -> 0x5008884AC4 (type=1) cfgGetAny: value=0x0 error=101 2023-06-30 19:46:32 ZWESVUSR DEBUG Ignoring environment variable: CEE_ENVFILE=DD:STDENV, conflict 2023-06-30 19:46:32 ZWESVUSR DEBUG shared env pos 0 is ZLDEBUG=ON 2023-06-30 19:46:32 ZWESVUSR DEBUG shared env pos 1 is CONFIG=/global/zowe/zowe.yaml 2023-06-30 19:46:32 ZWESVUSR DEBUG shared env pos 2 is ROOT_DIR=/usr/lpp/zowe 2023-06-30 19:46:32 ZWESVUSR DEBUG about to get workspace dir vdref object i=0 k=zowe vdref object i=1 k=workspaceDirectory 2023-06-30 19:46:32 ZWESVUSR DEBUG Directory '/global/zowe/workspace' exists 2023-06-30 19:46:32 ZWESVUSR INFO ZWEL0058I WORKSPACE_DIR is '/global/zowe/workspace' 2023-06-30 19:46:32 ZWESVUSR DEBUG about to prepare Zowe instance 2023-06-30 19:46:32 ZWESVUSR DEBUG about to run command 'CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe" /usr/lpp/zowe/bin/zwe internal start prepare --config "/global/zowe/zowe.yaml" --ha-instance {{ha_instance_id}} 2>&1' InternalError: invalid opcode (op=0 [invalid], pc=0) 2023-06-30 19:46:38 ZWESVUSR DEBUG command 'CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe" /usr/lpp/zowe/bin/zwe internal start prepare --config "/global/zowe/zowe.yaml" --ha-instance {{ha_instance_id}} 2>&1' ran successfully 2023-06-30 19:46:38 ZWESVUSR INFO ZWEL0018I Zowe instance prepared successfully 2023-06-30 19:46:38 ZWESVUSR DEBUG about to get component list 2023-06-30 19:46:38 ZWESVUSR DEBUG about to run command 'CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe"CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe" /usr/lpp/zowe/bin/zwe internal get-launch-components --config "/global/zowe/zowe.yaml" --ha-instance {{ha_instance_id}} 2>&1' 2023-06-30 19:46:44 ZWESVUSR DEBUG command 'CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe"CONFIG="/global/zowe/zowe.yaml" ROOT_DIR="/usr/lpp/zowe" /usr/lpp/zowe/bin/zwe internal get-launch-components --config "/global/zowe/zowe.yaml" --ha-instance {{ha_instance_id}} 2>&1' ran successfully 2023-06-30 19:46:44 ZWESVUSR ERROR ZWEL0029E start component list is empty

Additional context os: z/OS 2.4 nodejs: v16.17.1 java: Java(TM) SE Runtime Environment (build 8.0.8.0 - pmz3180sr8-20230314_01(SR8)) IBM J9 VM (build 2.9, JRE 1.8.0 z/OS s390-31-Bit 20230313_47323 (JIT enabled, AOT enabled) OpenJ9 - 4d03a8c OMR - 49526cb IBM - 7187a01) JCL - 20230208_01 based on Oracle jdk8u361-b09 zowe - zowe-2.10.0-staging-3375-20230630074043 (happens with all previous builds too)

[DivergentEuropeans]

With this zwe internal get-launch-components --config "FILE(/global/zowe/zowe.yaml)" while FILE is correct use in YAML, I'm not sure if this is the correct usage in zwe commands. Maybe try, zwe internal get-launch-components --config /global/zowe/zowe.yaml

I say this because, perhaps some part of the code accepts this syntax and doesn't fail immediately. ie 2023-06-30 19:46:38 ZWELNCH:33620899 ZWESVUSR DEBUG command 'CONFIG="/global/zowe/zowe.yaml" Other parts may be putting into parsing or logic sections that don't expect a format like this, which could unfortunately cause a confusing long error like this

[DivergentEuropeans]

@lucasortega99

[lucasortega99]

Hi @DivergentEuropeans,

using the syntax you mentioned also results in the same error: InternalError: ss_check bytecode buffer overflow (op=232 [if_false8], pc=7). Function=

Also, if i try the "zwe internal start prepare --config /global/zowe/zowe.yaml" on the terminal, i get the following, which is also on the started task output: InternalError: invalid opcode (op=0 [invalid], pc=0)

[JoeNemo]

Has anyone in Zowe development ever reproduced this? This is similar to a bug that we have seen on some mainframes that is very hard to reproduce. There is either a corruption or resource issue that has been lurking for months.

[ccc13481]

Hi

I have this one right now - with or with out configmgr=false.

Is there anything I can do to help debug this?

BR Frank Allan Rasmussen Danske Bank

[Joe-Winchester]

A customer is reporting that they also get the invaid opcode error, although in this case Zowe works OK and it's the UMS server that causes the exception. Customer is willing to accept a patch to collect more trace/diagnostic. Mentioned this on the z/OS squad scrum call Oct 18th.

 2023-09-15 13:17:32 <ZWELNCH:33816765> C009232 DEBUG about to run command 'CONFIG="PARMLIB(SYS3.IZP.PARMLIB(ZWEYAML)):FILE(/var/zowe/GS-zowe-2.11.0.yaml)" ROOT_DIR="/usr/lpp/zowe/zowe-2.11.0" /usr/lpp/zowe/zowe-2.11.0/bin/zwe internal start prepare --config "PARMLIB(SYS3.IZP.PARMLIB(ZWEYAML)):FILE(/var/zowe/GS-zowe-2.11.0.yaml)" --ha-instance ÊÊha_instance_idÂÂ 2>&1'
 InternalError: invalid opcode (op=0 ›invalid®, pc=0)

[nosrednayduj]

So the same customer (at least, their data sets and filenames in the messages are the same) got back to us in the UMS team, where they haven't said anything on the ticket for several months, mentioning that they only get the error with useConfigmgr: true. Which is sort of vacuous. But it suggests that they want us to reply. Is there anything we can ask them to do to help diagnose this problem? Have there been any updates to config manager in later Zowe versions, so we could ask them to upgrade and see if it goes away? (customer appears to have 2.11)

[ccc13481]

Hi

It also dies in 2.14:

2024-03-05 22:37:42.089 C009232 INFO ZWEL0021I Zowe Launcher 2024-03-05 22:37:42.091 C009232 INFO -- If you cant see '500' 2024-03-05 22:37:42.103 C009232 INFO ZWEL0023I Zowe YAML conf 2024-03-05 22:37:42.103 C009232 INFO ZWEL0024I HA_INSTANCE_ID 2024-03-05 22:37:42.106 C009232 INFO ZWEL0017I ROOT_DIR is '/ 2024-03-05 22:37:42.110 C009232 INFO ZWEL0069I Configuration 2024-03-05 22:37:42.111 C009232 INFO ZWEL0058I WORKSPACE_DIR InternalError: invalid opcode (op=0 [invalid], pc=0)

2024-03-05 22:37:46.487 C009232 INFO ZWEL0018I Zowe instance 2024-03-05 22:37:50.800 C009232 ERROR ZWEL0029E start compone

BR Frank Allan Rasmussen Danske Bank

[JoeNemo]

@nosrednayduj can you reproduce with any regularity?

[ccc13481]

@JoeNemo I just need to set useConfigmgr to true and I will get the error. If you could instruct me to setup some debugging I would be happy to do so.

BR Frank Allan Rasmussen Danske Bank

[nosrednayduj]

@JoeNemo Unfortunately, I personally have never seen this. It's only at the customer site. So what we need is some idea of what to tell them to do for debugging. Is there any verbose settings we can give them that might give us more information? Is there anything we can ask them about their environment that might give us ideas of where the thing could have gone wrong?

[ccc13481]

Hi

Are there a 'compile zwelnch for dummies' guide?

I'm stuck on a CRLF-problem in build.sh.

If I can get it to compile, I will put printf statements in to see if I can pinpoint the place of the error.

BR Frank Allan Rasmussen Danske Bank

[JoeNemo]

@nosrednayduj , please find me to set up a review meeting. This needs attention.

[ccc13481]

OK - I got ZWELNCH compiled and I will test to see if I can pinpoint where the error occur.

BR Frank Allan Rasmussen Danske Bank

[ccc13481]

OK - did some basic tests:

C009232 INFO ZWEL0021I Zowe Launcher starting
C009232 INFO -- If you cant see '500' at the end of the line, your log is too short to read!80--------90------ 100----------------------125------------------- C009232 INFO ZWEL0023I Zowe YAML config file is 'FILE(/var/zowe/GS-zowe-2.14.0.yaml)'
C009232 INFO ZWEL0024I HA_INSTANCE_ID is 'ha_instance_id'
C009232 INFO ZWEL0017I ROOT_DIR is '/usr/lpp/zowe/zowe-2.14.0'
C009232 INFO ZWEL0069I Configuration is valid
C009232 INFO MAIN 1
C009232 INFO ZWEL0058I WORKSPACE_DIR is '/tmp/zowe/workspace'
C009232 INFO MAIN 2
C009232 INFO PREP 1
C009232 INFO PREP 2
C009232 INFO RUN 1 'ROOT_DIR="/usr/lpp/zowe/zowe-2.14.0" /usr/lpp/zowe/zowe-2.14.0/bin/zwe internal start prepare --config "/var/zowe/GS-zowe-2.14.0.yaml" --h C009232 INFO RUN 2 '1c15b2b8'
C009232 INFO RUN 3 '1024'
C009232 INFO RUN 4 'InternalError: invalid opcode (op=0 [invalid], pc=0)

d], pc=0)
C009232 INFO RUN 4 '

C009232 INFO RUN 5
C009232 INFO PREP 3
C009232 INFO ZWEL0018I Zowe instance prepared successfully
C009232 INFO MAIN 3
C009232 INFO RUN 1 'ROOT_DIR="/usr/lpp/zowe/zowe-2.14.0"ROOT_DIR="/usr/lpp/zowe/zowe-2.14.0" /usr/lpp/zowe/zowe-2.14.0/bin/zwe internal get-launch-components C009232 INFO RUN 2 '1c15b2b8'
C009232 INFO RUN 3 '1024'
C009232 INFO RUN 4 'js_load_file failure. A file requested module ' ' but it could not be loaded.

C009232 INFO RUN 4 'Getting file info for diagnosis

C009232 INFO RUN 4 'Could not get file info for .js. status=-1, rc=129, rsn=87752812

C009232 INFO RUN 4 'ReferenceError: @ @ }l

C009232 INFO RUN 4 '

C009232 INFO RUN 5
C009232 ERROR ZWEL0029E start component list is empty

The interesting code is here (around line 1380 in main.c) : static int run_command(const char command, handle_line_callback_t handle_line, void data) { DEBUG("about to run command '%s'\n", command);

INFO(FRRA_RUN_1, command);

FILE *fp = popen(command, "r");

INFO(FRRA_RUN_2, fp);

if (!fp) {
ERROR(MSG_CMD_RUN_ERR, command, strerror(errno));
return -1;
}
char *line;
char buf[1024] = {0};

INFO(FRRA_RUN_3, sizeof(buf));

while((line = fgets(buf, sizeof(buf) - 1, fp)) != NULL) {

INFO(FRRA_RUN_4, line);                                                                  

handle_line(data, line);                                                                 
memset(buf, '\0', sizeof(buf));                                                          

}
if (ferror(fp)) {
pclose(fp);
ERROR(MSG_CMD_OUT_ERR, command, strerror(errno));
return -1;
}
int rc = pclose(fp);
if (rc == -1) {
ERROR(MSG_CMD_RUN_ERR, command, strerror(errno));
} else if (rc > 0) {
WARN(MSG_CMD_RCP_WARN, command, rc);
return -1;
}

INFO(FRRA_RUN_5);

DEBUG("command '%s' ran successfully\n", command);
return 0;
}

The error seems to be in a "fgets" that gets a strange error.

I tried to put in the following code:

fp = popen(command, "r");
INFO(FRRA_RUN_2, fp);
line = fgets(buf, sizeof(buf) - 1, fp);
INFO(FRRA_RUN_4, line);
pclose(fp);

and I get the same error C009232 INFO RUN 2 '1c15b2b8'
C009232 INFO RUN 4 'InternalError: invalid opcode (op=0 .invalid., pc=0)

BR Frank Allan Rasmussen Danske Bank

[ccc13481]

Hi

The error was because of an LE runtime option in our environment.

HEAPPOOLS64 must be OFF.

This is set in quite a few file.

A fix should be in the works.

Frank Allan Rasmussen Danske Bank

[1000TurquoisePogs]

v2.16 will contain HEAPPOOLS64(OFF) everywhere I found occurrences of runopts specified, so I believe this issue is resolved. If a person is running an older Zowe and sees "InternalError: invalid opcode...", it seems you would have to set HEAPPOOLS64(OFF) within the parmlib that controls the runopts for that user, since otherwise there's about 25 of Zowe's files in which this change must be done.

You can confirm it is off by running OMVS RUNOPTS('RPTOPTS(ON)') in TSO as the user account which runs zowe.

I'm going to close this issue as I believe this is the fix for all users who have seen it so far, but feel free to open new issues if the error is seen again even in v2.16, or see it even when you know HEAPPOOLS64 is OFF.