search

zowe / docs-site

Documentation for the Zowe project
https://docs.zowe.org/
Creative Commons Attribution 4.0 International
70 stars 134 forks source link

ZWESLSTC failed to start #2315

Closed connorpekovic closed 2 years ago

connorpekovic commented 2 years ago

Description

Hi Zowe Team!

At our shop we have ZWESISTC started :tada:. We ran zwe start --config path/to/zowe.yaml to start ZWESLSTC and got this output in the console.

$ export PATH=${PATH}:/usr_lpp/zowe/bin                                         

$ zwe start --config /path/to/zowe.yaml                             
=============================================================================== 
>> STARTING ZOWE                                                                

  * Output:                                                                     
    node: read_yaml 10: /usr_lpp/zowe/bin/commands/start/index.sh 21: .: zwecli_
process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found                 
  * Output:                                                                     
    node: read_yaml 10: /usr_lpp/zowe/bin/commands/start/index.sh 22: .: zwecli_
process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found                 
  * Output:                                                                     
    node: read_yaml 10: /usr_lpp/zowe/bin/commands/start/index.sh 29: .: zwecli_
process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found                 
------------------------------------------------------------------------------- 
>> Job security_stcs_zowe is started successfully.                              

$

We got this output in the SYSPRINT.

2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0021I Zowe Launcher starting
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0023I Zowe YAML config  file is '/path/to/zowe.yaml'
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0024I HA_INSTANCE_ID is 'lpar1'
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0020I loading '/path/to/zowe.yaml'
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0017I ROOT_DIR is '/usr_lpp/zowe/bin'
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC INFO ZWEL0058I WORKSPACE_DIR is '/path/to/workspace'
2022-07-27 12:57:59 <ZWELNCH:50398258> ZWESLSTC ERROR ZWEL0030E failed to prepare Zowe instance

This isn't a documentation issue as much as it is my mainframe shop looking for support. Our shop completed every step in the documentation leading up to this step. I can supply more info as needed.

Pages to Update

Screenshots

Expected behavior

Expected the ZWESLSTC to start and stay up.

Additional context

Big thank you!

nannanli commented 2 years ago

@Joe-Winchester @jackjia-ibm @1000TurquoisePogs Hi all, do you have any suggestions or insights about the issue described above? Thank you!

jackjia-ibm commented 2 years ago

@connorpekovic I saw potentially 2 issues from the info you provided:

  1. ZWEL0017I ROOT_DIR is '/usr_lpp/zowe/bin', usually ROOT_DIR shouldn't point to the bin folder but the zowe root directory. Normally, this value should be /usr_lpp/zowe. You may double check the value of zowe.runtimeDirectory.
  2. node: read_yaml 10: /usr_lpp/zowe/bin/commands/start/index.sh 21: .: zwecli_process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found error is usually caused by cannot find node. Please double check if node.home is properly defined in zowe.yaml, including indentation format.

Please let us know if this solves the issues you have.

connorpekovic commented 2 years ago

@jackjia-ibm

Thank you! The team made a change and this is the new output in ZWESLSTC's SYSPRINT: The output in the Unix console was the same:

2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0021I Zowe Launcher starting 
2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0023I Zowe YAML config file is '/global/zowe/zowe200/zowe.yaml' 
2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0024I HA_INSTANCE_ID is 'lpar1' 
2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0020I loading '/global/zowe/zowe200/zowe.yaml' 
2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0017I ROOT_DIR is '/usr_lpp/zowe' 
2022-07-28 13:28:52 <ZWELNCH:50398198> ZWESLSTC INFO ZWEL0058I WORKSPACE_DIR is '/global/zowe/workspace' 
2022-07-28 13:28:54 <ZWELS:67175123> ZWESLSTC INFO (zwe-internal-start-prepare:337) Zowe version: v2.0.0 
2022-07-28 13:28:57 <ZWELS:67175123> ZWESLSTC INFO (zwe-internal-start-prepare:338) build and hash: v2.x/master#995 (0d8d5bf395041167f07bb047f34708e3ba261405) 
  * Output:       
    node: read_yaml 10: /usr_lpp/zowe/bin/commands/internal/start/prepare/index.sh 350: .: zwecli_process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found 
2022-07-28 13:28:58 <ZWELS:67175123> ZWESLSTC INFO (zwe-internal-start-prepare:354) starting Zowe instance lpar1 with /global/zowe/zowe200/zowe.yaml ... 
2022-07-28 13:28:58 <ZWELS:67175123> ZWESLSTC ERROR (zwe-internal-start-prepare,prepare_workspace_directory:19) WARNING: Failed to set permission of some existing files or directories in /global/zowe/workspace: 
2022-07-28 13:28:58 <ZWELS:67175123> ZWESLSTC ERROR (zwe-internal-start-prepare,prepare_workspace_directory:20) chmod: FSUM6180 file "/global/zowe/workspace": EDC5139I Operation not permitted. (errno2=0x00000004) 
node: generate_instance_env_from_yaml_config 15: prepare_workspace_directory 37: /usr_lpp/zowe/bin/commands/internal/start/prepare/index.sh 366: .: zwecli_process_command 4: /usr_lpp/zowe/bin/zwe 96: FSUM7351 not found 
2022-07-28 13:28:58 <ZWELS:67175123> ZWESLSTC ERROR (bin/libs/config.sh,generate_instance_env_from_yaml_config:19) ZWEL0140E: Failed to translate Zowe configuration (/global/zowe/zowe200/zowe.yaml). 
2022-07-28 13:28:58 <ZWELNCH:50398198> ZWESLSTC ERROR ZWEL0030E failed to prepare Zowe instance

Thank you so much! Connor

jackjia-ibm commented 2 years ago

@connorpekovic there are 2 issues from this log:

  1. node is still not found. There are few things to double check:
    • if node.home is defined in zowe.yaml, and if the format/indentation is good.
    • whether there is bin/node executable under the directory of node.home.
    • if node.home section is too complicated. Reading this value is using shell script. Sometimes if this section is too complicated, like multiple home: lines, etc, may confused the parser.
  2. ZWEL0140E: Failed to translate Zowe configuration is caused by permission failure on workspace directory. Will need to make sure that Zowe run time user (by default ZWESVUSR) has write permission on it.
connorpekovic commented 2 years ago

@jackjia-ibm @nannanli The ZWESLSTC has now started and stayed up!!

There's still work to done at our shop. We're at version 2.0 and the caching service is not starting because it's HLQ is in our SMPE zone, presumably because that's we set our zowe.setup.dataset.prefix to. We'll tackle that tomorrow morning.

We hope the Zowe Desktop comes up after the caching service is fixed, then we can begin validating our Zowe installation and then begin standing up UMS for z/OS to get IBM DB2 Admin Foundations.

Knamasiv commented 5 months ago

/u/users/zowe/usr/lpp/zowrtm/bin/commands/start/index.sh 21: .: zwecli_process_command 4: ./zwe 96: FSUM7351 not found Exit code: 255

14.40.36 STC32143 IFA104I REGISTRATION HAS BEEN DENIED FOR 072
072 PRODUCT WITH OWNER=IBM CORP NAME=z/OS
072 FEATURE=SDSF VERSION=.. ID=5650-ZOS

I received above error while starting zowe from USS, please suggest steps to fix this issue

I have node pointed to path /u/users/zowe/usr/lpp/IBM/cnj/v20r0/IBM/node-v20.11.0-os390-s390x-202402131732 in YAML file

Joe-Winchester commented 4 months ago

@joepun76 , do you have anything quick you can spot from the above log extract.

Knamasiv commented 4 months ago

Hi Joe, We received below error in the log.

2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0021I Zowe Launcher starting 2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0023I Zowe YAML config file is '/309C/u/users/zowe/zowe.yaml' 2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0024I HA_INSTANCE_ID is '309c' 2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0020I loading '/309C/u/users/zowe/zowe.yaml' 2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0017I ROOT_DIR is '/309C/u/users/zowe/usr/lpp/zowrtm' 2024-05-08 16:30:34 ZWESVUSR INFO ZWEL0058I WORKSPACE_DIR is '/309C/u/users/zowe/workspace' 2024-05-08 16:31:58 ZWESVUSR INFO (zwe-internal-start-prepare:337) Zowe version: v2.0.0 2024-05-08 16:32:06 ZWESVUSR INFO (zwe-internal-start-prepare:338) build and hash: v2.x/master#995 (0d8d5bf395041167f07bb047f34708e3ba261405)

ZWESVUSR id have been given superuser authority UID(0)

joepun76 commented 4 months ago

Looks like node product reg SMF 89…you can set your .profile to use node v20 do node -v to see if you get the version # back.

Is it a new zowe install.

On Thu, May 9, 2024 at 10:50 AM Joe Winchester @.***> wrote:

@joepun76 https://github.com/joepun76 , do you have anything quick you can spot from the above log extract.

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2102815807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7N6T3XXUCP2IDUAOBJTZBOELNAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJQGI4DCNJYGA3Q . You are receiving this because you were mentioned.Message ID: @.***>

joepun76 commented 4 months ago

How do you install Zowe? From pax?

On Thu, May 9, 2024 at 11:12 AM Knamasiv @.***> wrote:

Hi Joe, We received below error in the log.

2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0021I Zowe Launcher starting 2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0023I Zowe YAML config file is '/309C/u/users/zowe/zowe.yaml' 2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0024I HA_INSTANCE_ID is '309c' 2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0020I loading '/309C/u/users/zowe/zowe.yaml' 2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0017I ROOT_DIR is '/309C/u/users/zowe/usr/lpp/zowrtm' 2024-05-08 16:30:34 ZWELNCH:84476275 ZWESVUSR INFO ZWEL0058I WORKSPACE_DIR is '/309C/u/users/zowe/workspace' 2024-05-08 16:31:58 ZWELS:34144821 ZWESVUSR INFO (zwe-internal-start-prepare:337) Zowe version: v2.0.0 2024-05-08 16:32:06 ZWELS:34144821 ZWESVUSR INFO (zwe-internal-start-prepare:338) build and hash: v2.x/master#995 (0d8d5bf395041167f07bb047f34708e3ba261405)

  • Output: node: read_yaml 10: /309C/u/users/zowe/usr/lpp/zowrtm/bin/commands/internal/start/prepare/index.sh 350: .: zwecli_process_command 4: /309C/u/users/zowe/usr/lpp/zowrtm/bin/zwe 96: FSUM7351 not found 2024-05-08 16:32:15 ZWELS:34144821 ZWESVUSR INFO (zwe-internal-start-prepare:354) starting Zowe instance 309c with /309C/u/users/zowe/zowe.yaml ... node: generate_instance_env_from_yaml_config 15: prepare_workspace_directory 37: /309C/u/users/zowe/usr/lpp/zowrtm/bin/commands/internal/start/prepare/index.sh 366: .: zwecli_process_command 4: /309C/u/users/zowe/usr/lpp/zowrtm/bin/zwe 96: FSUM7351 n 2024-05-08 16:32:25 ZWELS:34144821 ZWESVUSR ERROR (bin/libs/config.sh,generate_instance_env_from_yaml_config:19) ZWEL0140E: Failed to translate Zowe configuration (/309C/u/users/zowe/zowe.yaml). 2024-05-08 16:32:26 ZWELNCH:84476275 ZWESVUSR ERROR ZWEL0030E failed to prepare Zowe instance

ZWESVUSR id have been given superuser authority UID(0)

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2102857339, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7N6D65FRCR6SKGUZLETZBOG7VAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJQGI4DKNZTGM4Q . You are receiving this because you were mentioned.Message ID: @.***>

Knamasiv commented 4 months ago

Yes its new zowe install. I did through SMPE. Now I can start ZWESLSTC task but receiving error IEF403I ZWESLSTC - STARTED - TIME=12.59.34
+ZWEL0021I Zowe Launcher starting
$HASP708 ZWESLSTC CEEOPTS OPEN FAILED
RC=03 DATA SET ALREADY OPENED
DSNAME=ZWESVUSR.ZWESLSTC.STC00296.D0000101.?
IEC141I 013-C0,IGG0199G,ZWESLSTC,ZWESLSTC,CEEOPTS 927
ERROR DESCRIPTION: IEC141I
An open failure occurred for a subsystem data set.Possible failures
follow:

ZWESLSTC JCL: //ZWESLSTC PROC RGN=0M //ZWELNCH EXEC PGM=ZWELNCH,REGION=&RGN,TIME=NOLIMIT,
// PARM='ENVAR(_CEE_ENVFILE=DD:STDENV)/&HAINST.'
//STEPLIB DD DSNAME=SYSP.GT.ZOWE.SZWEAUTH,DISP=SHR
//SYSIN DD DUMMY
//SYSPRINT DD SYSOUT=,LRECL=1600
//SYSERR DD SYSOUT=
//STDOUT DD SYSOUT=
//STDERR DD SYSOUT=
//CEEOPTS DD
POSIX(ON)
//STDENV DD
CONFIG=/309C/u/users/zowe/zowe.yaml
/*

joepun76 commented 4 months ago

What version of zowe are you installing? If you use SMP/E, you need to apply PTFs depending on the version you want and run zwe init?

The proc you used looks like a pretty old one,

Main differences are (current one in below)...

//ZWELNCH EXEC PGM=ZWELNCH,REGION=&RGN,TIME=NOLIMIT, // PARM='ENVAR(_CEE_ENVFILE=DD:STDENV),POSIX(ON)/&HAINST.'

and

//STDENV DD _CEE_ENVFILE_CONTINUATION=\ _CEE_RUNOPTS=HEAPPOOLS(OFF) _EDC_UMASK_DFLT=0002 CONFIG=/SYSTEM/var/lpp/zowe/v2/zowe.yaml /

You do not need the //CEEOPTS DD * POSIX(ON)

SMP/E install starts with the base Zowe V2.0 and you need to APPLY the latest PTFs or the version you planned...

On Mon, May 13, 2024 at 2:48 PM Knamasiv @.***> wrote:

Yes its new zowe install. I did through SMPE. Now I can start ZWESLSTC task but receiving error IEF403I ZWESLSTC - STARTED - TIME=12.59.34 +ZWEL0021I Zowe Launcher starting $HASP708 ZWESLSTC CEEOPTS OPEN FAILED RC=03 DATA SET ALREADY OPENED DSNAME=ZWESVUSR.ZWESLSTC.STC00296.D0000101.? IEC141I 013-C0,IGG0199G,ZWESLSTC,ZWESLSTC,CEEOPTS 927 ERROR DESCRIPTION: IEC141I An open failure occurred for a subsystem data set.Possible failures follow:

  • A SYSIN or SYSOUT data set opened by JES.
  • A SYSIN or SYSOUT data set opened under the master subystem.
  • A subsystem data set(such as an HFS data set) that uses the same interface as JES.
  • For JES2, a SYSOUT data set incurred a subsystem JCL failure(SJF), and JES2 might issue a $HASP708 message to further describe the error. For JES3, a reserved DD name, such as JOURNAL, might have been specified in a DD statement. In either case, the failing DCB was not opened; processing for other DCBs opened in parallel continues normally.

ZWESLSTC JCL: //ZWESLSTC PROC RGN=0M //ZWELNCH EXEC PGM=ZWELNCH,REGION=&RGN,TIME=NOLIMIT, // PARM='ENVAR(_CEE_ENVFILE=DD:STDENV)/&HAINST.' //STEPLIB DD DSNAME=SYSP.GT.ZOWE.SZWEAUTH,DISP=SHR //SYSIN DD DUMMY //SYSPRINT DD SYSOUT= ,LRECL=1600 //SYSERR DD SYSOUT= //STDOUT DD SYSOUT= //STDERR DD SYSOUT= //CEEOPTS DD POSIX(ON) //STDENV DD CONFIG=/309C/u/users/zowe/zowe.yaml /*

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2108566569, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7N6XR44SATD7PWMOORDZCEDGZAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJQHA2TMNRVGY4Q . You are receiving this because you were mentioned.Message ID: @.***>

Knamasiv commented 4 months ago

Thanks Joe. Yes we applied latest PTF and current version of Zowe is 2.15. I made change to the Started task JCL and didn't get any error related to CEEOPTS.

joepun76 commented 4 months ago

Great!

BTW, after applying PTFs, suggest running below 2 so you do not need to update manually. Thx.

Run 'zwe init mvs -c zowe.yaml --allow-overwrite’

Run ‘zwe init stc -c zowe.yaml --allow-overwrite’

On Tue, May 14, 2024 at 11:28 AM Knamasiv @.***> wrote:

Thanks Joe. Yes we applied latest PTF and current version of Zowe is 2.15. I made change to the Started task JCL and didn't get any error related to CEEOPTS.

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2110533278, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7NZN2R4MRJDOHVTYM6DZCIUTTAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGA2TGMZSG44A . You are receiving this because you were mentioned.Message ID: @.***>

Knamasiv commented 4 months ago

Thanks Joe. I am getting below error in ZSS appserver log

ZWES1060W Failed to init TLS environment, rc=1(Handle is not valid)
ZWES1065E Failed to configure https server, check agent https settings

Knamasiv commented 4 months ago

In Appserver log, I could see below error

ZWED0159E - Could not read zowe manifest, version unknown

joepun76 commented 4 months ago

I seen this error in my working instance, ignore it pls.

On Tue, May 14, 2024 at 4:36 PM Knamasiv @.***> wrote:

In Appserver log, I could see below error

ZWED0159E - Could not read zowe manifest, version unknown

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2111096941, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7NYXZC3JWVIUYIYPKLLZCJYU3AVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGEYDSNRZGQYQ . You are receiving this because you were mentioned.Message ID: @.***>

joepun76 commented 4 months ago

I need to see your zowe.yaml if I can continue.

Is zowe up and running?

On Tue, May 14, 2024 at 4:27 PM Knamasiv @.***> wrote:

Thanks Joe. I am getting below error in ZSS appserver log

ZWES1060W Failed to init TLS environment, rc=1(Handle is not valid) ZWES1065E Failed to configure https server, check agent https settings

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2111082802, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7N2KHORHBRPLJSDU3D3ZCJXRVAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGEYDQMRYGAZA . You are receiving this because you were mentioned.Message ID: @.***>

Knamasiv commented 4 months ago

ZWESLSTC task is up with errors related to HTTPS Could you please provide your email-id to send YAML file

joepun76 commented 4 months ago

@.*** pls. and the zowe STC JOB OUTPUT pls. Thanks.

On Wed, May 15, 2024 at 9:39 AM Knamasiv @.***> wrote:

ZWESLSTC task is up with errors related to HTTPS Could you please provide your email-id to send YAML file

— Reply to this email directly, view it on GitHub https://github.com/zowe/docs-site/issues/2315#issuecomment-2112565594, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQIU7N5M5KHNVN3BAALVODTZCNQQJAVCNFSM543BEGJ2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGI2TMNJVHE2A . You are receiving this because you were mentioned.Message ID: @.***>

Knamasiv commented 4 months ago

Hi Joe, Please share your email-id