ansible-collections / ibm_zos_core

Red Hat Ansible Certified Content for IBM Z
75 stars 44 forks source link

[Bug] [zos_operator] Evaluate the ability to escape command prefixes #1444

Open fernandofloresg opened 2 months ago

fernandofloresg commented 2 months ago

Is there an existing issue for this?

Bug description

Had this ansible to issue cmd

    - name: "{{nodeType }} STC running check using cmd"
      delegate_to: zceeZosLpar
      ibm.ibm_zos_core.zos_operator:
        cmd: "$D JQ''{{ lookup('vars', inventory_hostname).nodeName|upper }}''"
      register: bc2Result   

Note the double single quotes around the {{ and }}

A year ago this ran all ok, using older version of zOS Ansible, as shown by this output from then

TASK [ocpzcx-stc-status : bootstrap STC running check cmd result] ******************************************************************************
Friday 10 February 2023  20:46:24 +0000 (0:00:03.525)       0:01:31.173 ******* 
 [0;32mok: [ocpboot] => { [0m
 [0;32m    "msg": [ [0m
 [0;32m        "ZZZZ      2023041  15:46:23.33             ISF031I CONSOLE ZCXPRV1 ACTIVATED", [0m
 [0;32m        "ZZZZ      2023041  15:46:23.33            -$D JQ'OCPDBOOT'", [0m
 [0;32m        "ZZZZ      2023041  15:46:23.33  STC08176   $HASP890 JOB(OCPDBOOT)", [0m
 [0;32m        "                                           $HASP890 JOB(OCPDBOOT)  STATUS=(AWAITING HARDCOPY),CLASS=STC,", [0m
 [0;32m        "                                           $HASP890                PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)", [0m
 [0;32m        "ZZZZ      2023041  15:46:23.33  STC08819   $HASP890 JOB(OCPDBOOT)", [0m
 [0;32m        "                                           $HASP890 JOB(OCPDBOOT)  STATUS=(EXECUTING/ZZZZ),CLASS=STC,", [0m
 [0;32m        "                                           $HASP890                PRIORITY=15,SYSAFF=(ZZZZ),HOLD=(NONE)", [0m
 [0;32m        "", [0m
 [0;32m        "", [0m
 [0;32m        "Ran 0 \"$D JQ''OCPDBOOT''\" QUIET WAIT" [0m
 [0;32m    ] [0m

Can see cmd only ended up with single quotes so it works ok 

Now using this level of Ansible

ibm.ibm_zos_core 1.8.0
ibm.ibm_zosmf    1.4.1

Ran it today, but it failed

In zOS see

BPXF024I (OMVSKERN) Apr 10 04:18:10 WTZZZZ ansible-ibm.ibm_zos_core.zo   
 969                                                                     
s_job_query: Invoked with job_name=OCPDBOOT owner=None job_id=None       
IEF450I ZCXPRV18 STEP1 - ABEND=S0C4 U0000 REASON=00000004 970            
        TIME=04.18.15                                                    
IEF450I ZCXPRV19 STEP1 - ABEND=S0C4 U0000 REASON=00000004 971            
        TIME=04.18.22                                                    
BPXF024I (OMVSKERN) Apr 10 04:18:23 WTZZZZ ansible-ibm.ibm_zos_core.zo   
 972                                                                     
s_operator: Invoked with cmd=$D JQ''OCPDBOOT'' verbose=False             
wait_time_s=1                                                            
BPXF024I (OMVSKERN) Apr 10 04:18:23 WTZZZZ ansible-ibm.ibm_zos_core.zo   
 973                                                                     
s_operator: Invoked with cmd=$D JQ''OCPDBOOT''                           
IEA630I  OPERATOR ZCXP0000 NOW ACTIVE,   SYSTEM=ZZZZ    , LU=ZCXPRV1     
JQ''OCPDBOOT''                                                           
IEE305I JQ''OCPD COMMAND INVALID                                         
IEA631I  OPERATOR ZCXP0000 NOW INACTIVE, SYSTEM=ZZZZ    , LU=ZCXPRV1     
BPXF024I (OMVSKERN) Apr 10 04:18:27 WTZZZZ ansible-ibm.ibm_zos_core.zo   
 978                                                                     
s_find: Invoked with patterns=['OCPLDS.OCPDBOOT.ROOT'] resource_type=c   
luster age_stamp=creation_date age=None contains=None excludes=None      
size=None pds_patterns=None volume=None                                  

from above, see cmd is: $D JQ''OCPDBOOT'' 

if issue that from SDSF it does fail

Changed ansible, removing one of the double quotes

   - name: "{{nodeType }} STC running check using cmd"
      delegate_to: zceeZosLpar
      ibm.ibm_zos_core.zos_operator:
        cmd: "$D JQ'{{ lookup('vars', inventory_hostname).nodeName|upper }}'"
      register: bc2Result

after change still fails, see

BPXF024I (OMVSKERN) Apr 10 04:26:01 WTZZZZ ansible-ansible.legacy.copy  
 079                                                                    
: Invoked with src=/u/zcxprv1/.ansible/tmp/ansible-tmp-1712737559.8006  
68-12027-86366500943514/source dest=/global/zcx/ocpdemo/ocpdboot.prope  
rties mode=None follow=False _original_basename=ocp-template-workflow_  
variables.openshift.properties checksum=1f188f5ad6981ab2e4d1c5fcf36d9f  
86d635d680 backup=False force=True unsafe_writes=False content=NOT_LOG  
GING_PARAMETER validate=None directory_mode=None remote_src=None        
local_follow=None owner=None group=None seuser=None serole=None         
selevel=None setype=None attributes=None                                
BPXF024I (OMVSKERN) Apr 10 04:26:02 WTZZZZ ansible-ibm.ibm_zos_core.zo  
 080                                                                    
s_encode: Invoked with src=/global/zcx/ocpdemo/ocpdboot.properties      
encoding={'from': 'ISO8859-1', 'to': 'IBM-1047'} backup=False           
backup_compress=False dest=None backup_name=None tmp_hlq=None           
BPXF024I (OMVSKERN) Apr 10 04:26:02 WTZZZZ ansible-ibm.ibm_zos_core.zo  
 081                                                                    
s_encode: Invoked with src=/global/zcx/ocpdemo/ocpdboot.properties      
encoding={'from': 'ISO8859-1', 'to': 'IBM-1047'}                        
BPXF024I (OMVSKERN) Apr 10 04:26:04 WTZZZZ ansible-ibm.ibm_zos_core.zo  
 082                                                                    
s_job_query: Invoked with job_name=OCPDBOOT owner=None job_id=None      
BPXF024I (OMVSKERN) Apr 10 04:26:11 WTZZZZ ansible-ibm.ibm_zos_core.zo  
 083                                                                    
s_operator: Invoked with cmd=$D JQ'OCPDBOOT' verbose=False              
wait_time_s=1                                                           
BPXF024I (OMVSKERN) Apr 10 04:26:11 WTZZZZ ansible-ibm.ibm_zos_core.zo  
 084                                                                    
s_operator: Invoked with cmd=$D JQ'OCPDBOOT'                            
IEA630I  OPERATOR ZCXP0000 NOW ACTIVE,   SYSTEM=ZZZZ    , LU=ZCXPRV1    
JQ'OCPDBOOT'                                                            
IEE305I JQ'OCPDB COMMAND INVALID                                        
IEA631I  OPERATOR ZCXP0000 NOW INACTIVE, SYSTEM=ZZZZ    , LU=ZCXPRV1    
BPXF024I (OMVSKERN) Apr 10 04:26:15 WTZZZZ ansible-ibm.ibm_zos_core.zo  

but that cmd entered via sdsf does work

$D JQ'OCPDBOOT'                                               
$HASP890 JOB(OCPDBOOT) 164                                    
$HASP890 JOB(OCPDBOOT)  STATUS=(EXECUTING/ZZZZ),CLASS=STC,    
$HASP890                PRIORITY=15,SYSAFF=(ZZZZ),HOLD=(NONE) 
$HASP890 JOB(OCPDBOOT) 165                                    
$HASP890 JOB(OCPDBOOT)  STATUS=(AWAITING HARDCOPY),CLASS=STC, 
$HASP890                PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)   

why is it so ?

IBM z/OS Ansible core Version

v1.8.0 (default)

IBM Z Open Automation Utilities

v1.2.5 (default)

IBM Enterprise Python

v3.11.x (default)

ansible-version

v2.16.x (default)

z/OS version

v2.5 (default)

Ansible module

zos_operator

Playbook verbosity output.

No response

Ansible configuration.

No response

Contents of the inventory

No response

Contents of group_vars or host_vars

No response

fernandofloresg commented 2 months ago

If is an easy fix, is a candidate for v1.9.1.

richp405 commented 2 months ago

Demetrios slack conversation: [CSLEJ8VGV/p1712820367812199?thread_ts=1712820340.578559&cid=CSLEJ8VGV]

fernandofloresg commented 2 months ago

Came back to read this carefully and looks like is just an escaping issue with the $ symbol, asked the user to try again.

ddimatos commented 2 months ago

I happened to be looking at something in the same space, originally I thought it might have to do with the single quotes but instead it really had to do with the command prefix $ not being escaped.

I think this issue should be scoped to multiple issues: 1) Documentation in the short term, where the docs might instruct users to escape command prefixes, eg $, # and the doc include some updated samples. One might be to show and example of how to figure out the command prefix the system is using and possibly mention this in documentation as well. For example, this command D OPDATA,PREFIX (short version I believe is D O) yields:

  00- 05.56.00           D OPDATA,PREFIX                                        
      05.56.00           IEE603I 05.56.00 OPDATA DISPLAY 634                  C 
       PREFIX     OWNER      SYSTEM     SCOPE     REMOVE   FAILDSP              
       $          JES2       EC33017A   SYSTEM    NO       SYSPURGE             
       MQMA       VCA        EC33017A   SYSTEM    NO       PURGE                
       MQMB       VCB        EC33017A   SYSTEM    NO       PURGE                
       MQM1       VC1        EC33017A   SYSTEM    NO       PURGE                
       MQM2       VC2        EC33017A   SYSTEM    NO       PURGE                
       MQM3       VC3        EC33017A   SYSTEM    NO       PURGE                
       MQM4       VC4        EC33017A   SYSTEM    NO       PURGE                
       MQM5       VC5        EC33017A   SYSTEM    NO       PURGE                
       MQM6       VC6        EC33017A   SYSTEM    NO       PURGE                
       MQM7       VC7        EC33017A   SYSTEM    NO       PURGE                
       MQM8       VC8        EC33017A   SYSTEM    NO       PURGE                
       MQM9       VC9        EC33017A   SYSTEM    NO       PURGE                
       REXX7A     AXR        EC33017A   SYSPLEX   NO       PURGE 

Showing the JES2 command prefix is $, read more about command prefix in next bullet (2) below.

2) A second issue to evaluate the ability to escape command prefixes. This could get a bit complex as JES2 commands use a command prefix, documentation shows it as being a $ but it can be configured to be other special chars, this will need research, you can start by reviewing this section of the doc about Table 2. Command Syntax Conventions

Because your installation establishes the JES2 command identifier, it may be some character other than $. If your installation's command identifier is not specified in your initialization stream by CONCHAR= parameter on the CONDEF statement, the $ is the default.This publication shows the format of a command entered through any console. A command entered through a card reader has a /* (slash asterisk) in card columns 1 and 2 preceding the command identifier.

If the module offers escaping, it will get more complex than the command prefix, research will need to be done for Apostrophes, brackets, parentheses, etc , see Table 2. Command Syntax Conventions

3) A follow on to bullet (2) , MVS System commands, see Table 1. System command summary which don't necessarily require a command prefix , see System command formats allow for special characters in the operator commands such as ' # $ & ( ) * + , - . / ¢ < | ! ; ¬ % _ > ? : @ " =, see MVS system commands reference , so additional research would need to be done if the module can escape these as well without altering the commands purpose. I suspect there will be no issue escaping but testing and research will need to be done, I can't think of a MVS system command offhand with a $ but one with equals is D M=CPU.

Here is a python successful execution if a similar JES2 command with single quotes.

>>> from zoautil_py import opercmd
>>> from zoautil_py.exceptions import ZOAUException
>>> print(opercmd.execute(command="\$D J\'SSHSTRT\'"))
[ZOAUResponse]
    rc: 0
    response_format: UTF-8
    stdout_response: EC33017A   2024113  12:43:35.00             ISF031I CONSOLE OMVS0000 ACTIVATED
EC33017A   2024113  12:43:35.00            -$D J'SSHSTRT'
EC33017A   2024113  12:43:35.00             $HASP890 JOB(SSHSTRT)
                                           $HASP890 JOB(SSHSTRT)   STATUS=(AWAITING HARDCOPY),CLASS=A,
                                           $HASP890                PRIORITY=1,SYSAFF=(ANY),HOLD=(NONE)

    stderr_response:
    command: opercmd -T 0 -- "\$D J'SSHSTRT' "

This also worked, it depicts not needing to escape single quotes:

 print(opercmd.execute(command="\$D J'SSHSTRT'"))
EdwardMcCarthy commented 2 months ago

I originally came across this issue

some thoughts...

  1. ideally the solution should have a a way to have the cmd passed where no escaping is done, so don't get into these escaping traps, maybe that's not possible in ansible, but this would be the preferred resolution

  2. perhaps have a new module - jes2_cmd , where you pass the jes2 cmd without the initial $ and then the code auto adds the $ or as there are other subsystems that you can issue cmds with a leading char - I think MQ can do this - maybe a new module - ssi_cmd - with a parm to set what the initial char and a parm for the rest of the command

  3. or change tso_command to pass the in a new parm the command prefix which then get concatenated in front of the cmd supplied

ddimatos commented 2 months ago

These are all good suggestions @EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness.

The second issue (this one) will be to evaluate our options of which some are:

EdwardMcCarthy commented 2 months ago

Hi

Appreciate it’s a tricky issue.

Re - This would mean users don't have to escape anything, but would add overhead to the execution time and possibly regular module maintenance

I’ not sure I would be too worried about this adding to the execution time. In that says this add a few extra seconds to make or not making escaping decisions, so what, is anyone really going to notice/care ?

The regular maint issue I couldn’t comment on since I don’t write/maintain the code 😉

Re - MVS commands which won't be escaped

There might be zOS cmds like this

F stcid,parm=abc$/%zyx

That is some cmds for products might have a value for a parm that has chars like $, %, etc that may or may not need to be escaped. Offhand I cannot think of such an example right now.

Regards

Edward McCarthy

Senior zStack Technical Specialist Asia/Pacific IBMers Value Dedication to every client's success Innovation that matters - for the company and for the world

Trust and personal responsibility in all relationships Location Code: CAGR

Phone: +61 2 6212 1137 Fax: +61 2 6124 2155

Mobile: +61 411 254 783

Email / Sametime: @.**@.>

To infinity and beyond

From: Demetri @.> Sent: Tuesday, April 30, 2024 3:24 AM To: ansible-collections/ibm_zos_core @.> Cc: EDWARD MCCARTHY @.>; Mention @.> Subject: [EXTERNAL] Re: [ansible-collections/ibm_zos_core] [Bug] [zos_operator] Evaluate the ability to escape command prefixes (Issue #1444)

These are all good suggestions @EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness. The second issue (this one) will be to evaluate our options of which some are: Consider additional ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization. Report Suspicious https://us-phishalarm-ewt.proofpoint.com/EWT/v1/AdhS1Rd-!-XFRWZ5ZeETxVl8SZW8KO4UajSUZK3aTO_xIPkoYdec2D4kUek3ApX2OGVElA7W_Q5yreQLBzAWz9N20ZfSOI7wjnglka314DoxIpQENTZXJgbh1SzZbx7yJiRcB$ ‌ ZjQcmQRYFpfptBannerEnd

These are all good suggestions @EdwardMcCarthyhttps://github.com/EdwardMcCarthy , the issue is being split into 2 issues, first is to use doc to address and bring awareness.

The second issue (this one) will be to evaluate our options of which some are:

— Reply to this email directly, view it on GitHubhttps://github.com/ansible-collections/ibm_zos_core/issues/1444#issuecomment-2083265138, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APAPVRYKO6PHLRVP3AEYJTDY7Z64NAVCNFSM6AAAAABGDMWFY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBTGI3DKMJTHA. You are receiving this because you were mentioned.Message ID: @.**@.>>

ddimatos commented 2 weeks ago

Thanks Edward, we will take into consideration the comment:

MVS commands which won't be escaped

There might be zOS cmds like this

F stcid,parm=abc$/%zyx