[Bug] The copy of a load module from PDSE to a PDSE failed

jbyibm commented 1 year ago

Bug description

When we try to copy a load module from PDSE to a PDSE zos_copy failed with this message.

fatal: [zos_host_01]: FAILED! => {"changed": false, "failed_when_result": true, "msg": "Unable to copy data set member NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01) to data set member NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)", "rc": 1, "stderr": null, "stderr_lines": null, "stdout": "cp: FSUM8976 Error writing \"//'NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)'\" to PDSE member \"//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)'\".: EDC5000I No error occurred.\ncp: !//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)': close failed: EDC5091I The requested function could not be performed because a system utility failed.\n", "stdout_lines": null}

Playbook verbosity output

TASK [ibm.ibm_zos_wazi_deploy.zos_deploy : [deploy_modules>update>member_archive]: Copy NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01) to NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)] *** fatal: [zos_host_01]: FAILED! => {"changed": false, "failed_when_result": true, "msg": "Unable to copy data set member NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01) to data set member NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)", "rc": 1, "stderr": null, "stderr_lines": null, "stdout": "cp: FSUM8976 Error writing \"//'NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)'\" to PDSE member \"//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)'\".: EDC5000I No error occurred.\ncp: !//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)': close failed: EDC5091I The requested function could not be performed because a system utility failed.\n", "stdout_lines": null}

Contents of `ansible.cfg`

[defaults]
forks = 25
host_key_checking = False
#callbacks_enabled = timer, profile_tasks, profile_roles
stdout_callback = ibm.ibm_zos_wazi_deploy.cb_evidences

[ssh_connection]
pipelining = True
#ssh_args = -o ControlMaster=auto -o ControlPersist=3600s -o PreferredAuthentications=publickey

Contents of the inventory

No response

Contents of `group_vars` or `host_vars`

No response

Ansible version

ansible [core 2.11.12]
  config file = None
  configured module search path = ['/home/jby/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /home/jby/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True

IBM z/OS Ansible core Version

v1.4.0

IBM ZOAU version

v1.2.1

z/OS version

2.4

Ansible module

zos_copy

jbyibm commented 1 year ago

In this line https://github.com/ansible-collections/ibm_zos_core/blob/7a5bb929646bb130a70970edec82f0e882870861/plugins/modules/zos_copy.py#L1361 in our config err is empty but out have:

cp: FSUM8976 Error writing "//'NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)'" to PDSE member "//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)'".: EDC5000I No error occurred.
cp: !//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)': close failed: EDC5091I The requested function could not be performed because a system utility failed.

So if I replace err with out it works

fernandofloresg commented 1 year ago

Hi Jean, thank you for the feedback and suggestion. Will look into this

jbyibm commented 1 year ago

Thanks, strange behavior... Maybe it depends on LPAR, ZOAU version, ...

fernandofloresg commented 1 year ago

Could you share the playbook used? This error message "Unable to copy data set member NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01) to data set member NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)" confuses me since you are trying to copy a USS file, is the file named " NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)"?

jbyibm commented 1 year ago

Internally, I did this modification:

if ("FSUM8976" in err and "EDC5091I" in err) or ("FSUM8976" in out and "EDC5091I" in out) :

jbyibm commented 1 year ago

Sorry, wrong title

jbyibm commented 1 year ago

fatal: [zos_host_01]: FAILED! => {
    "changed": false,
    "failed_when_result": true,
    "invocation": {
        "module_args": {
            "backup": false,
            "dest": "NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)",
            "force": true,
            "is_binary": true,
            "remote_src": true,
            "src": "NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)"
        }
    },
    "msg": "Unable to copy data set member NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01) to data set member NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)",
    "rc": 1,
    "stderr": null,
    "stderr_lines": null,
    "stdout": "cp: FSUM8976 Error writing \"//'NAZARE.WDEPLOY.CORE140.LOADLIB(LGIPOL01)'\" to PDSE member \"//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)'\".: EDC5000I No error occurred.\ncp: !//'NAZARE.WDEPLOY.CORE140.BACK.LOADLIB(LGIPOL01)': close failed: EDC5091I The requested function could not be performed because a system utility failed.\n",
    "stdout_lines": null
}

As you can see the message is in stdout

jbyibm commented 1 year ago

I think this topic has been covered in a thread before, but I couldn't find it (and the related issue). I don't have much experience with z/OS. However, in my interactions with DBB and customers, the basic command used to copy a load module to a PDS(e) is cp -X -I. Basically what I usually do:

cp -X -I For the copy of load modules (USS<->PDS).
cp -F bin For the copy of binaries, DBRM for example (USS<->PDS)
cp For the copy of text files, JCL for example (USS<->PDS)

The module zos_copy use the ZOAU dcp I tried several use case. ####### First test: ####### Here is what is called from ZOAU (called by zos_copy ) or manually

PDS Creation (with zos_data_set) dtouch -tLIBRARY -s10CYL -e20CYL -rU -l0 NAZARE.WDEPLOY.ANSIBLE.LOAD01

Load Module Copy (with zos_copy) dcp -B /SYSTEM/tmp/LGIPOL01 'NAZARE.WDEPLOY.ANSIBLE.LOAD01(LGIPOL01)'

Manual cp from a shell dcp -X /SYSTEM/tmp/LGIPOL01 'NAZARE.WDEPLOY.ANSIBLE.LOAD01(LGIPOL01)'

Step 3 failed with this error:

IDI0001I Fault Analyzer V14R1M16 (UI80002 2022/03/30) invoked by IDIXDCAP using USER.PARMLIB(IDICNF00)
IDI0002I Module FSUMUSSH offset X'7494C': Abend S0F4-X'10'

IDI0003I Fault ID F01720 assigned in history file TOOLS.IDI.HIST
cp: FSUMF140 IEWBIND function "SAVEW" failed with return code 16 and reason code 83ee2900
If failed because in step 2 dcp is called with -B  (I would say instead of -X )

########## Second test: ########## Here is what is called from ZOAU (called by zos_copy ) or manually

PDS Creation (manual from a shell or it can be an existing customer PDS) dtouch -tLIBRARY -s10CYL -e20CYL -rU -l0 NAZARE.WDEPLOY.ANSIBLE.LOAD02

Load Module Copy (manual from a shell) dcp -X /SYSTEM/tmp/LGIPOL01 'NAZARE.WDEPLOY.ANSIBLE.LOAD02(LGIPOL01)'

Load Module Copy (with zos_copy) dcp -B /SYSTEM/tmp/LGIPOL01 'NAZARE.WDEPLOY.ANSIBLE.LOAD02(LGIPOL01)'

Step 3 will not failed only if I patch this line in zos_copy : https://github.com/ansible-collections/ibm_zos_core/blob/7a5bb929646bb130a70970edec82f0e882870861/plugins/modules/zos_copy.py#L1361. See: https://github.com/ansible-collections/ibm_zos_core/issues/601

This gives me the impression that something is missing in zos_copy. We should have one more attribute like is_load to use the correct option for dcp?

is_load -> dcp -X -I is important for many customers but I does not work for me.
is_binary -> dcp -B
not is_load and not is_binary -> dcp

fernandofloresg commented 1 year ago

Hi Jean, when having this issue, are both PDSE load libs?

jbyibm commented 1 year ago

yes

ddimatos commented 1 year ago

@fernandofloresg since you just delivered this for support/1.4.1 you could probably cherrypick the squashed commit into branches based on dev: git checkout <some base branch> git pull git checkout -b some/new/branch git push -u origin some/new/branch git cherry-pick a871dd74f5b085906f386cbccdb6a80aefbea2d4 --no-commit

Reason for --no-commit is because you will get the changelog fragment which will reflect the PR you did for 1.4.1 , thus with no-commit it gives you a chance to edit it with a new commit. its a bit annoying that the changelog fragements lock us into a model we can't just do a plain cherry-pick, maybe we should rethink the fragment convention a bit.

ansible-collections / ibm_zos_core