suite_db2_setup_for_manage role fails while creating MIRRORLOGPATH directory

[SalaryTheft]

I've set the MAS_CONFIG_SCOPE variable to system (See #1056). I'm also using NFS storage class provisioned with kubernetes-sigs/nfs-subdir-external-provisioner

TASK [ibm.mas_devops.suite_db2_setup_for_manage : Creating MIRRORLOGPATH folder in c-db2w-shared-db2u-0] **************************************************************************
fatal: [localhost]: FAILED! =>
{"changed": true, 
"cmd": "oc exec -it -n db2u c-db2w-shared-db2u-0 -- su -lc \"mkdir -p /mnt/backup/MIRRORLOGPATH\" db2inst1\n", 
"delta": "0:00:00.384900", "end": "2023-10-18 21:28:10.371843", "msg": "non-zero return code", "rc": 1, "start": "2023-10-18 21:28:09.986943", 
"stderr": "Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)\nUnable to use a TTY - input is not a terminal or the right kind of file\nmkdir: cannot create directory ‘/mnt/backup’: Permission denied\ncommand terminated with exit code 1", 
"stderr_lines": ["Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)", "Unable to use a TTY - input is not a terminal or the right kind of file", "mkdir: cannot create directory ‘/mnt/backup’: Permission denied", "command terminated with exit code 1"], 
"stdout": "", "stdout_lines": []}

[SalaryTheft]

I extracted suite_db2_setup_for_manage role from 18.2.0 and pasted it to latest version and it helped. Seems like it is related to #917.

[whitfiea]

@csschen can you look at this issue please as it occurrs due to those setup_db2 changes

[csschen]

I have tested the role suite_db2_setup_for_manage with version 18.4.1 on both IBMC and AWS, both of them run successfully.

The DB2 instance version is 11.5.8.0-cn2.

Below are the test log. db2_manage_setup_roks.log db2_manage_setup_rosa.log

It looks like the problem is NFS storage, didn't test on the storage provisioned with kubernetes-sigs/nfs-subdir-external-provisioner before, need to do further investigate. FYI @SalaryTheft @whitfiea

[SalaryTheft]

I've just tested with LVM storage and getting same error. Here are all the variables exported for the DB2/Maximo Manage installation, just in case.

export MAS_APP_SETTINGS_SECONDARY_LANGS='KO'
export MAS_APP_SETTINGS_DEMODATA=true
export MAS_APP_ID=manage
export MAS_APPWS_COMPONENTS="base=latest,health=latest"
export MAS_APP_SETTINGS_PERSISTENT_VOLUMES_FLAG=true

export MAS_APP_SETTINGS_DOCLINKS_PVC_STORAGE_CLASS=lvms-vg1
export MAS_APP_SETTINGS_BIM_PVC_STORAGE_CLASS=lvms-vg1
export MAS_APP_SETTINGS_JMS_QUEUE_PVC_STORAGE_CLASS=lvms-vg1
export DB2_CPU_REQUESTS=1
export DB2_MEMORY_REQUESTS=4Gi
export DB2_META_STORAGE_ACCESSMODE=ReadWriteOnce # LVM Storage does not support RWX
export DB2_DATA_STORAGE_CLASS=lvms-vg1
export DB2_META_STORAGE_CLASS=lvms-vg1

ansible-playbook ibm.mas_devops.oneclick_add_manage

[csschen]

We need enhance the Ansible role suite_db2_setup_for_manage to support NFS storage and LVM storage for now. Is there any fyre cluster I can use to setup the environment for testing? @whitfiea

[SalaryTheft]

I am still getting same error and had to do the same workaround. Any progress?

TASK [ibm.mas_devops.suite_db2_setup_for_manage : Creating MIRRORLOGPATH folder in c-db2w-shared-db2u-0] ***************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "oc exec -it -n db2u c-db2w-shared-db2u-0 -- su -lc \"mkdir -p /mnt/backup/MIRRORLOGPATH\" db2inst1\n", "delta": "0:00:00.435653", "end": "2024-01-19 11:47:38.155762", "msg": "non-zero return code", "rc": 1, "start": "2024-01-19 11:47:37.720109", "stderr": "Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)\nUnable to use a TTY - input is not a terminal or the right kind of file\nmkdir: cannot create directory ‘/mnt/backup’: Permission denied\ncommand terminated with exit code 1", "stderr_lines": ["Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)", "Unable to use a TTY - input is not a terminal or the right kind of file", "mkdir: cannot create directory ‘/mnt/backup’: Permission denied", "command terminated with exit code 1"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT *****************************************************************************************************************************************

PLAY RECAP *************************************************************************************************************************************************
localhost                  : ok=82   changed=17   unreachable=0    failed=1    skipped=37   rescued=0    ignored=0

Here is my current NFS server configuartion - just in case.

[admin@bastion nfs]$ ls -alh
total 4.0K
drwxr-xr-x.  4 root root   32 Jan 19 00:45 .
drwxr-xr-x.  3 root root   17 Jan 17 17:28 ..
drwxr-xr-x. 21 root root 4.0K Jan 19 11:33 pv
drwxr-xr-x.  2 root root    6 Jan 17 09:10 registry
[admin@bastion nfs]$ cat /etc/exports
/disk1/nfs/pv       *(rw,no_subtree_check,sync,no_wdelay,insecure,no_root_squash)
/disk1/nfs/registry *(rw,no_subtree_check,sync,no_wdelay,insecure,no_root_squash)

[SalaryTheft]

Same thing is happening with the latest playbook. Directory permissions and NFS export options are the same as described in the comment above
(let me know if those settings are wrong).

TASK [ibm.mas_devops.suite_db2_setup_for_manage : Creating MIRRORLOGPATH folder in c-db2w-shared-db2u-0] *****************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "oc exec -it -n db2u c-db2w-shared-db2u-0 -- su -lc \"mkdir -p /mnt/backup/MIRRORLOGPATH\" db2inst1\n", "delta": "0:00:00.394695", "end": "2024-07-11 14:31:07.127984", "msg": "non-zero return code", "rc": 1, "start": "2024-07-11 14:31:06.733289", "stderr": "Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)\nUnable to use a TTY - input is not a terminal or the right kind of file\nmkdir: cannot create directory ‘/mnt/backup’: Permission denied\ncommand terminated with exit code 1", "stderr_lines": ["Defaulted container \"db2u\" out of: db2u, init-labels (init), init-kernel (init)", "Unable to use a TTY - input is not a terminal or the right kind of file", "mkdir: cannot create directory ‘/mnt/backup’: Permission denied", "command terminated with exit code 1"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT *******************************************************************************************************************************************

PLAY RECAP ***************************************************************************************************************************************************
localhost                  : ok=90   changed=15   unreachable=0    failed=1    skipped=65   rescued=0    ignored=0

[SalaryTheft]

While doing a database backup with db2 role, I ran into a similar issue.

TASK [ibm.mas_devops.db2 : Get storage usage of pvc temporary folder] **********
fatal: [localhost]: FAILED! => {"changed": false, "cmd": "oc exec c-db2w-shared-db2u-0 -c db2u -n db2u -- su -lc ' mkdir -p /mnt/backup/db2-db2w-shared-full-20240715163448; df -h /mnt/backup/db2-db2w-shared-full-20240715163448 ' db2inst1\n", "delta": "0:00:00.478478", "end": "2024-07-15 16:34:58.368502", "msg": "non-zero return code", "rc": 1, "start": "2024-07-15 16:34:57.890024", "stderr": "mkdir: cannot create directory ‘/mnt/backup’: Permission denied\ndf: /mnt/backup/db2-db2w-shared-full-20240715163448: No such file or directory\ncommand terminated with exit code 1", "stderr_lines": ["mkdir: cannot create directory ‘/mnt/backup’: Permission denied", "df: /mnt/backup/db2-db2w-shared-full-20240715163448: No such file or directory", "command terminated with exit code 1"], "stdout": "", "stdout_lines": []}

NO MORE HOSTS LEFT *************************************************************

PLAY RECAP *********************************************************************
localhost                  : ok=86   changed=2    unreachable=0    failed=1    skipped=76   rescued=0    ignored=0

No problem occurs when not specifying the user db2inst1 at the end of the command.

[admin@localhost ~]$ oc exec c-db2w-shared-db2u-0 -c db2u -n db2u -- su -lc ' mkdir -p /mnt/backup/db2-db2w-shared-full-20240715161306; df -h /mnt/backup/db2-db2w-shared-full-20240715161306 '
Filesystem      Size  Used Avail Use% Mounted on
overlay         200G  113G   88G  57% /