Separate downloaded copybooks / includes according to Workspace

[FALLAI-Denis]

Description of the enhancement requested

IBM Z Open Editor relies on Zowe Explorer for downloading and storing copybooks / includes retrieved from the z/OS mainframe.

This storage is unique and common to all active Workspaces.

This does not allow working on different versions of copybooks / includes depending on the Workspace, or even on the Git branch if the Workspace is associated with a Git Repository.

It would therefore be necessary:

• either an evolution of Zowe Explorer to be able to declare a storage folder relating to the Workspace,
• or that Z Open Editor adds information specific to the Workspace, or even specific to the Git branch in the Workspace, to the storage path.

See Need zowe.files.temporaryDownloadsFolder.path setting to be relative to current Workspace on zowe / vscode-extension-for-zowe Github site.

[phaumer]

@FALLAI-Denis could you provide specific scenarios where problem could happen?

• If you have your copybooks on MVS then changing branches or workspaces would not impact what you pull from MVS. How do you switch versions on MVS? Wouldn't they be in different data sets? Do you have examples where this could be a problem? You can use different Zowe profiles for different workspaces as the "zopeneditor.zowe" settings can be scoped to a workspace and each profile will get its own temp sub-folder.
• If you have you copybooks local then it will open the local files directly and not place them in a temp directory; so switching branches means accessing the files from that branch. Are you saying that you have copybooks in a different git repo in one workspace and two other workspaces access them? Then why not clone the copybook repo twice and adjust you property-groups temporarily or use our new ZAPP variables that can also be defined in the workspace settings to dynamically modify the ZAPP file?

[FALLAI-Denis]

Hi @phaumer,

In fact, being able to manage a storage path for downloaded copybooks based on workspace context is a necessary condition for versioning, but it is not a sufficient condition. You also need a way to define the search for copybooks / includes according to the workspace context, or even the Git branch if this workspace is a Git repository. To date, we must manually modify the declarations in the zapp.yaml file.

In discussion Interactions between the Git workflow and Builds actions, I put a diagram which describes the promotion of elements including copybooks / includes and which shows the use of a copybooks PDS concatenation mechanism at build time. Note on this diagram the existence of two promotion circuits: one for the evolutions, one for the corrections.

Since then we have evolved our solution and the application copybooks / includes are managed by cloned Git repositories in a dedicated folder in the workspace, with a mechanism that allows switching branches depending on the context of the workspace (monitoring of the .git folder at the root of the workspace, to extract the type / name of branch used, and synchronization by checkout of the copybooks / includes Git repositories cloned in a dedicated folder according to a configuration file which associates working branch of the workspace and branches in the copybooks / includes Git repositories). On the other hand, technical copybooks / includes continue to be managed by means of a PDS concatenation, a concatenation which depends on the context of the Git branch used in the workspace. This therefore allows us to have a versioned management of copybooks / includes and to use the local search of zapp.yaml, and therefore no use of the storage folder for elements downloaded from the remote z/os system. See Implementation of a solution compatible with IBM Z Open Editor for the management of compilation options and the search for COPY/INCLUDE.

Despite using this solution for application-related copybooks / includes, we also need to manage technical copybooks / includes that are managed only on z/OS systems, and where possible manage them in a versioned way. For this we use PDS which correspond to various levels of certification of these copybooks: in test, in integration, in user acceptance, in production.

We also have another need concerning the search for copybooks / includes which is linked to the fact that our applications can be multi z/os partitions: part of the application is intended for a z/os partition, and another part of the same application can be intended for a another z/os partition, but both parts must be managed globally and cohesively. For historical reasons, each part of the application is built on a related z/os partition, which requires having a management of downloaded copybooks / includes per z/os partition for the same workspace, with potentially the same copybook / include names on each of the partition. So we have a need to user more than one Zowe profile to achieve download of copybooks / includes.

I also refer you to another request made in connection with this subject: Need to have contextual property groups (relative to Git branches) .

There are probably other needs that may require having a relative path for storing downloaded copybooks/includes; versioning is just one of these use cases.

Objectively, there is no reason to mix artifacts (unique and common download folder) related to different work contexts (many Workspace).

[FALLAI-Denis]

Hi,

About location of copybooks, see also requirements for "SonarLint for COBOL":

By default, SonarLint takes the analysis configuration from the SonarQube or SonarCloud server therefore it is required that your project has already been analyzed by SonarQube or SonarCloud. The following COBOL analyzer properties are synced by default unless previously overridden locally. Note that all properties found on the server will be synced locally, not just this selection:

• sonar.cobol.dialect
• sonar.cobol.file.suffixes
• sonar.cobol.sourceFormat
• sonar.cobol.copy.suffixes
• sonar.cobol.copy.directories

In case copybooks are in different location locally, the analyzer property sonar.cobol.copy.directories should be defined in the /project/.vscode/settings.json file.

If working with COBOL files via Zowe explorer, it is recommended to update your Zowe workspace settings in VS Code by modifying the temporary file location; temporary files should be saved to your project folder which is bound to SonarQube or SonarCloud. With the correct configuration, the analysis will be executed normally and you should see detected problems.

Clearly, sonar.cobol.copy.directories setting must correspond to the location used by Z Open Editor for storing copybooks downloaded from mvs, (which is not the same folder that Zowe Explorer uses to open a PDS file/member from mvs), but also to the various locations declared in the zapp.yaml file(s).

[FALLAI-Denis]

Hi,

I suggest that the copybooks/includes download location be specified in the zapp.yaml file.

In principle each Workspace, for us Git Repository, has its own zapp.yaml file. This zapp.yaml file could be different from one Workspace version to another (from one Git branch to another if the zapp.yaml file is declared in the .gitignore file).

This would allow each Workspace to use its own versions of copybooks, without interference between the different Workspaces (but increasing the volume of data since there would then be duplication of copybooks/includes in each Workspace).

This would perfectly meet our different needs. Indeed, we use a .imports folder declared in .gitignore to import the different necessary copybooks by cloning Git repositories (Git repositories dedicated to the management of copybooks, separated from the Git repositories of the programs). It would then be enough for Z Open Editor to download the mvs copybooks into a subfolder of this .imports folder.

works consistent with the active git branch, (part after feature/) :

Example of implementation via the zapp.yaml file: For us should be a subfolder of .imports, like ".imports/downloaded".

Thanks.

[FALLAI-Denis]

Hi,

I read changelog for Zowe Explorer V3: it will manage mainframe files in memory (resource URI), instead of in local temp directory on disk (resource path).

ZOE use ZE services to download MVS Copybooks / Includes.

Does this address our need to have a "per workspace" management of downloaded Copybooks / Include?