HA: shared USS file system and VSAM data set configuration

[jackjia-ibm]

Is your feature request related to a problem? Please describe.

The feature supports the overall Zowe HA plan. This feature is relevant for Sysplex deployment.

Describe the solution you'd like

• Define requirements on setup shared USS file system and VSAM data set for Sysplex.
• Define method to encrypt VSAM data set (maybe using ICF) -- needs further investigation.
• Document the requirement.
• Make changes on Zowe instance configuration step.

Additional context

The VSAM data set will be used by Caching API.

[MarkAckert]

Related to api-layer#863

[jackjia-ibm]

This could be a way to define shared folder in sysplex: In this way,

/LPAR1/var/zowe/instance/instance.env and /LPAR2/var/zowe/instance/instance.env can be different, but /LPAR1/var/zowe/instance/shared/instance.env and /LPAR2/var/zowe/instance/shared/instance.env will be same because the shared mountpoint is using same ZFS.

[jackjia-ibm]

Additional configuration on VSAM data set, we need to limit its access to Zowe user/group only.

[jackjia-ibm]

VSAM

Zowe will use the KSDS VSAM data cluster. KSDS most closely resembles the actual cache as it is a way to store key/value pairs and effectively retrieve the value based on the key. Other reasons for using VSAM are described here. One good example setup for VSAM is Walmart’s zECS which is open-sourced at https://github.com/walmartlabs/zECS.

VSAM will be used via a Java library available in the z/OS environment. The example of usage is available at this link.

To prevent the limitation of keys of the same size, we will create a hash of the key and store the value under the cache. We will use the master key to differentiate the services and alternative key to differentiate the key/value pairs. This will allow us to retrieve all key/value pairs for a specific service. To prevent losing information contained in the key, we will store a JSON in the format of {“key”: “keyValue”, “value”: “valueValue”}

Prerequisites Open Question

What are the requirements of the VSAM set up in the system?

1. When we define or use a VSAM data set, we should consider how to share or access a data set within a single system and among multiple systems (Sysplex). Sometimes applications only need to read the data set. Sometimes an application needs to update a data set while other applications are reading it. The most complex case of sharing a VSAM data set is when multiple applications need to update the data set and all require complete data integrity.

• VSAM RLS (Record-Level Sharing)
  • Allows concurrent access in a sysplex to the existing VSAM data sets
  • Can provide full data integrity (read and write)
  • Serialization is done at the Record Level
  • RLS uses Coupling Facility-based locking and data caching to provide sysplex-scope locking and data access integrity
  • Record locks for UPDATE are always exclusive. Record locks for read depend on the level of read integrity.
  • z/OS Sysplex configuration is required to run VSAM RLS https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.idas200/vsamrls.htm
• VSAM share options
  • SHAREOPTIONS parameter is specified in DEFINE CLUSTER command
  • SHAREOPTIONS (cross-region,cross-system)
  • cross-region: the amount of sharing allowed between regions within the same system or multiple systems
  • cross-system: how the data set is shared between systems
  • https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.idai200/da6i2118.htm
  • The serialization can be done by GRS
  • Ex: SHAREOPTIONS(3,y), the data set can be opened by any number of users/applications for R/W request. User/application is responsible for maintaining data integrity.

2. VSAM data set encryption

   This encryption can be done through SAF controls and functions along with SMS policies. Encrypted data sets must be in SMS-managed extended format. ICSF can be used to encrypt the data.

(Copied from Jakub's doc https://docs.google.com/document/d/1Z8T1mq7MRYzxREL2pf1zTlj0tP7fqaR2ba31TZHX0Qw/edit#)

[jackjia-ibm]

Since we couldn't mount shared under instance directory of each lpar, we have to put the shared folder into /global and allow the user to customize it. Here is the suggested folders for Zowe installation:

# shared runtime (ZOWE_ROOT_DIR/ROOT_DIR)
/usr/lpp/zowe

# shared extension runtime (ZWE_EXTENSION_DIR)
/global/zowe/extension

# shared instance configs (ZOWE_INSTANCE_DIR/INSTANCE_DIR)
/global/zowe/instance/zowe.yaml (or)
/global/zowe/instance/instance.env - 
    ZWE_DISCOVERY_SERVICES_LIST=https://tivlp13:7553/eureka,https://tivlp14:7553/eureka
# other shared instance folders
/global/zowe/instance/logs
/global/zowe/instance/workspace/api-mediation/static-defs
/global/zowe/instance/workspace/app-server/plugins

# shared keystore
/global/zowe/keystore

[jackjia-ibm]

Details of setup VSAM are posted https://github.com/zowe/api-layer/issues/889#issuecomment-710626292. Pending work: limit data set access to ZWESVUSR.