search

IBM / zopeneditor-about

IBM Z Open Editor: File issues here!
https://ibm.github.io/zopeneditor-about
Apache License 2.0
47 stars 25 forks source link

zapp file does not find copybooks ( cobol ) #413

Closed kraabrsg closed 3 weeks ago

kraabrsg commented 1 month ago

Development environment used

Problem Description

Detailed steps for reproducing the problem:

When opening a cobol file not all of the copybooks get parsed, it is rather random which copybooks are found. A correct zapp file is placed in the first folder, in a multiroot workspace.

zapp file

{
    "name": "vmd Dev",
    "description": "cobol copybooks",
    "version": "3.0.0",
    "author": {
        "name": "Cobold TUX"
    },
    "propertyGroups": [
        {
            "name": "cobol-copybooks",
            "language": "cobol",
            "libraries": [
                {
                    "name": "syslib",
                    "type": "local",
                    "locations": [
                        "../.vscode/INC"
                    ]
                },
                {
                    "name": "syslib",
                    "type": "mvs",
                    "locations": [
                        "MYHOST.COPYBOOK"
                    ]
                }
            ]
        }
    ]
}

debug log:

This log file is available at C:\mypath2\data\user-data\logs\20240514T070600\window4\exthost\IBM.zopeneditor\zopeneditor-20240514-063430.log
Initialized IBM Z Open Editor logger on level "DEBUG".
2024-05-14T08:34:30.356+02:00 DEBUG: TelemetryStats after restart: {"lastResetDate":"2024-05-14T05:06:07.115Z","numberOfEvents":10298,"numberOfMaxEvents":0}
2024-05-14T08:34:30.369+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"activated","properties":{"editor":"Visual Studio Code"}}
2024-05-14T08:34:30.654+02:00 INFO : Zowe Explorer was modified for IBM Remote System Explorer API (RSE API) support by IBM Z Open Editor.
2024-05-14T08:34:30.810+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"licenses/state","properties":{"event.value":"trial-active","editor":"Visual Studio Code"}}
2024-05-14T08:34:30.835+02:00 DEBUG: ZCodeFormat: No zcodeformat file was found in C:\mypath\myfolder1. Formatter settings in other workspaces or the default formatter settings will be used. Started watcher for when a zcodeformat file does get created.
2024-05-14T08:34:30.836+02:00 DEBUG: ZCodeFormat: No zcodeformat file was found in C:\mypath\myfolder2. Formatter settings in other workspaces or the default formatter settings will be used. Started watcher for when a zcodeformat file does get created.
2024-05-14T08:34:31.079+02:00 DEBUG: ZAPP: A valid ZAPP file was found in "C:\mypath\myfolder1" and will be used instead of user settings.
2024-05-14T08:34:31.082+02:00 DEBUG: ZAPP: No ZAPP file was found in C:\mypath\myfolder2. User and workspace settings will be used. Started watcher for when a zapp file does get created.
2024-05-14T08:34:31.343+02:00 INFO : IBM Z Open Editor extension has (re)started.
2024-05-14T08:34:31.412+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"java/found","properties":{"editor":"Visual Studio Code","platform":"win32","java.majorversion":"17","java.vendor":"openjdk"},"measurements":{"sinceActivated":1.056}}
2024-05-14T08:34:31.414+02:00 DEBUG: JavaFinder: Used the "JAVA_HOME" environment variable to locate Java.
2024-05-14T08:34:31.702+02:00 DEBUG: JavaFinder: Used the "JAVA_HOME" environment variable to locate Java.
2024-05-14T08:34:31.736+02:00 DEBUG: ZLanguageServer.buildServerOptions: Starting COBOL language server from "language-server".
2024-05-14T08:34:31.821+02:00 DEBUG: ZLanguageServer: Will start a language server with command "C:\mypath3\jdk\17.0.2\bin\java.exe -Declipse.application=com.ibm.wazi.lsp.cobol.core.languageServer,-Dosgi.bundles.defaultStartLevel=4,-Declipse.product=com.ibm.wazi.lsp.cobol.core.languageServerProduct,-Xmx512m,-jar,C:\mypath2\data\extensions\ibm.zopeneditor-4.1.0\language-server\plugins\org.eclipse.equinox.launcher_1.6.400.v20210924-0641.jar,-configuration,C:\mypath2\data\extensions\ibm.zopeneditor-4.1.0\language-server\cobol_config_win,-data,C:\Users\myuser\AppData\Local\Temp\_ws_1715668471821".
2024-05-14T08:34:31.824+02:00 DEBUG: AbstractZLanguageClient.initializeLanguageClient: Started initializing a new COBOL language client with these initialization options: {"enableUnreachableCodeWarnings":true,"publishDiagnostics":true,"includeFileExtensions":[".cpy",".copybook",".copy",".cobcopy"],"compilerOptions":""}.
2024-05-14T08:34:31.871+02:00 DEBUG: AbstractZLanguageClient.startLanguageClient: Language client for COBOL started.
2024-05-14T08:34:32.032+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"zowe/profileTypes","properties":{"event.value":"zosmf","editor":"Visual Studio Code"}}
2024-05-14T08:34:32.160+02:00 DEBUG: Z Open Editor environment summary:
- Z Open Editor version: 4.1.0
- Editor platform: Visual Studio Code
- Editor platform version: 1.89.0
- Required minimal editor platform version: ^1.81.0
- Editor installation root directory: C:\mypath2\resources\app
- Editor installation id: 60c996423c869f575fa4dd8886b167f8e6525788fbf5d7c1ffba8b0a0d0ed9ce
- Editor workspaces: C:\mypath\myfolder1,C:\mypath\myfolder2
- Is a valid Java configured: true
  - Java: C:\mypath3\jdk\17.0.2\bin\java.exe
  - Java memory allocation: 512
  - Java startup user settings: Not defined.
- Key user settings:
  - Unreachable Code enabled: true
  - HLASM version used: 3.1
  - HLASM macros file used: 
  - HLASM instructions file used: C:\mypath2\data\extensions\ibm.zopeneditor-4.1.0\resources\hlasm-instructions.json
  - HLASM Macro resolution enabled: false
  - Enforce Case Sensitive Include File Names: false
  - ZAPP schema used: C:\mypath4/data/extensions/ibm.zopeneditor-4.1.0/resources/zapp-schema-1.1.0.json
  - Variables defined by all ZAPP documents in workspace: 
  - Found the following ZAPP variable declarations in VS Code settings: []
  - Automatic preprocessor execution setting: off
  - IBM User Build enabled: false
  - Zowe CLI API profile management used: v2 team config in use.
  - Zowe CLI Secure Credentials enabled: false
  - Zowe data providers used: zosmf
  - Zowe CLI profiles of types rse, zosmf or ssh found: entw-zosmf(zosmf),ssh(ssh)
  - Zowe CLI files profile name specified in VS Code user settings: Not defined.
  - Zowe CLI SSH profile name specified in VS Code user settings: Not defined.
  - Zowe CLI files profile used for remote include files: entw-zosmf
  - Zowe CLI SSH profile used: ssh
  - Zowe CLI profiles configured for user build: entw-zosmf (zosmf)
  - Zowe Explorer RSE API Mappings File setting: Not defined.
  - Maximum number of parallel downloads over Zowe profiles: 5
  - Include file download strategy: Directly tries to download files from data sets.
  - Zowe Temp-Dir value: Not defined.
  - COBOL Property Groups defined: cobol-copybooks
  - COBOL include file extensions: .cpy,.copybook,.copy,.cobcopy
  - COBOL disabled problems: false
  - PL/I Property Groups defined: 
  - PL/I include file extensions: 
  - PL/I disabled problems: true
  - HLASM Property Groups defined: 
  - HLASM include file extensions: .s,.assemble,.asm,.hlasm,.hla,.copyasm,.macasm,.bms,.mac
  - HLASM disabled problems: false
  - REXX disabled problems: false
  - VS Code Files Associations: *.COBOL*:cobol,*.COB*:cobol,*.COPYBOOK*:cobol,*.ASM*:hlasm,*.ASSEMBLE*:hlasm,*.HLASM*:hlasm,*.HLA*:hlasm,*.MACLIB*:hlasm,*.EXEC*:rexx,*.REXX*:rexx,*.REXXINC*:rexx,*.rex:rexx,*.rexx:rexx,*.COBCOPY*:cobol,*.COPY*:cobol
- Zowe Explorer VS Code Extension version: 2.15.4
  - Successfully activated: true
  - Installation path: C:\mypath2\data\extensions\zowe.vscode-extension-for-zowe-2.15.4
  - Log file location: C:\mypath2\data\extensions\zowe.vscode-extension-for-zowe-2.15.4\logs
- Other VS Code extensions visible: vscode.bat(1.0.0),vscode.clojure(1.0.0),vscode.coffeescript(1.0.0),vscode.configuration-editing(1.0.0),vscode.cpp(1.0.0),vscode.csharp(1.0.0),vscode.css(1.0.0),vscode.css-language-features(1.0.0),vscode.dart(1.0.0),vscode.debug-auto-launch(1.0.0),vscode.debug-server-ready(1.0.0),vscode.diff(1.0.0),vscode.docker(1.0.0),vscode.emmet(1.0.0),vscode.extension-editing(1.0.0),vscode.fsharp(1.0.0),vscode.git(1.0.0),vscode.git-base(1.0.0),vscode.github(0.0.1),vscode.github-authentication(0.0.2),vscode.go(1.0.0),vscode.groovy(1.0.0),vscode.grunt(1.0.0),vscode.gulp(1.0.0),vscode.handlebars(1.0.0),vscode.hlsl(1.0.0),vscode.html(1.0.0),vscode.html-language-features(1.0.0),vscode.ini(1.0.0),vscode.ipynb(1.0.0),vscode.jake(1.0.0),vscode.java(1.0.0),vscode.javascript(1.0.0),vscode.json(1.0.0),vscode.json-language-features(1.0.0),vscode.julia(1.0.0),vscode.latex(1.0.0),vscode.less(1.0.0),vscode.log(1.0.0),vscode.lua(1.0.0),vscode.make(1.0.0),vscode.markdown(1.0.0),vscode.markdown-language-features(1.0.0),vscode.markdown-math(1.0.0),vscode.media-preview(1.0.0),vscode.merge-conflict(1.0.0),vscode.microsoft-authentication(0.0.1),ms-vscode-remote.remote-wsl-recommender(0.0.19),ms-vscode.js-debug(1.89.0),ms-vscode.js-debug-companion(1.1.2),ms-vscode.vscode-js-profile-table(1.0.9),vscode.builtin-notebook-renderers(1.0.0),vscode.npm(1.0.1),vscode.objective-c(1.0.0),vscode.perl(1.0.0),vscode.php(1.0.0),vscode.php-language-features(1.0.0),vscode.powershell(1.0.0),vscode.pug(1.0.0),vscode.python(1.0.0),vscode.r(1.0.0),vscode.razor(1.0.0),vscode.references-view(1.0.0),vscode.restructuredtext(1.0.0),vscode.ruby(1.0.0),vscode.rust(1.0.0),vscode.scss(1.0.0),vscode.search-result(1.0.0),vscode.shaderlab(1.0.0),vscode.shellscript(1.0.0),vscode.simple-browser(1.0.0),vscode.sql(1.0.0),vscode.swift(1.0.0),vscode.theme-abyss(1.0.0),vscode.theme-defaults(1.0.0),vscode.theme-kimbie-dark(1.0.0),vscode.theme-monokai(1.0.0),vscode.theme-monokai-dimmed(1.0.0),vscode.theme-quietlight(1.0.0),vscode.theme-red(1.0.0),vscode.vscode-theme-seti(1.0.0),vscode.theme-solarized-dark(1.0.0),vscode.theme-solarized-light(1.0.0),vscode.theme-tomorrow-night-blue(1.0.0),vscode.tunnel-forwarding(1.0.0),vscode.typescript(1.0.0),vscode.typescript-language-features(1.0.0),vscode.vb(1.0.0),vscode.xml(1.0.0),vscode.yaml(1.0.0),connor4312.esbuild-problem-matchers(0.0.3),dbaeumer.vscode-eslint(2.4.4),IBM.zopeneditor(4.1.0),mkloubert.vs-remote-debugger(3.0.0)
2024-05-14T08:34:32.198+02:00 INFO : IBM Z Open Editor extension has (re)started.
2024-05-14T08:34:38.695+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"document/didOpen","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"sinceActivated":8.339}}
2024-05-14T08:34:38.704+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"document/didClose","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"sinceActivated":8.348}}
2024-05-14T08:34:38.802+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"includes/count","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"number":7,"sinceActivated":8.446}}
2024-05-14T08:34:39.129+02:00 DEBUG: Received request to stop the pl1 language server.
2024-05-14T08:34:39.131+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"pl1"},"measurements":{"sinceActivated":8.775}}
2024-05-14T08:34:39.678+02:00 DEBUG: Received request to stop the pl1 language server.
2024-05-14T08:34:39.679+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"pl1"},"measurements":{"sinceActivated":9.323}}
2024-05-14T08:34:40.181+02:00 DEBUG: Received request to stop the pl1 language server.
2024-05-14T08:34:40.182+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"pl1"},"measurements":{"sinceActivated":9.826}}
2024-05-14T08:34:40.686+02:00 DEBUG: Received request to stop the pl1 language server.
2024-05-14T08:34:40.687+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"pl1"},"measurements":{"sinceActivated":10.331}}
2024-05-14T08:34:41.157+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"document/didOpen","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"sinceActivated":10.801}}
2024-05-14T08:34:46.012+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"includes/count","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"number":7,"sinceActivated":15.656}}
2024-05-14T08:34:46.016+02:00 DEBUG: Received request to stop the pl1 language server.
2024-05-14T08:34:46.017+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"pl1"},"measurements":{"sinceActivated":15.661}}
2024-05-14T08:34:49.957+02:00 DEBUG: Received request to stop all language servers.
2024-05-14T08:34:49.958+02:00 DEBUG: Received request to stop the cobol language server.
2024-05-14T08:34:49.958+02:00 DEBUG: UsageReporter: Sending the following telemetry data: {"eventName":"languageClient/stop","properties":{"editor":"Visual Studio Code","language":"cobol"},"measurements":{"sinceActivated":19.602}}
2024-05-14T08:34:49.961+02:00 DEBUG: IBM Z Open Editor extension has been deactivated for a restart.
2024-05-14T08:34:49.982+02:00 DEBUG: ZLanguagesResourceResolver.documentRequested: The COBOL language server requested the following include file: "ARZKMSGC". The ID for this request was assigned as "ybqrlt" There are 1 pending requests being processed.
2024-05-14T08:34:49.984+02:00 WARN : You opened a file with include files, but do not have a Property Group defined that specifies where to find them. Review the documentation for Zapp files.
2024-05-14T08:34:49.986+02:00 WARN : The COBOL language server requested the include file "ARZKMSGC", which was not found. enforceCaseSensitiveIncludeFileNames was set to false. Check your Zowe CLI profile and property group settings and if you have not already done so, switch to the "DEBUG" log level for more information.
2024-05-14T08:34:49.986+02:00 DEBUG: ZLanguagesResourceResolver.buildResponse(RequestId:ybqrlt): File "ARZKMSGC" was not found.
2024-05-14T08:34:49.987+02:00 DEBUG: ZLanguagesResourceResolver.buildResponse(RequestId:ybqrlt): Finished request ybqrlt. 0 pending requests left to process.
2024-05-14T08:34:49.995+02:00 DEBUG: Received request to stop the pl1 language server.

image

errors, also mentionend in other github issues: image

Files in the local folder are always found, but not the copybooks on the mainframe (after taking a look into this topic again, and it seems that the copybooks are missing because of ZoweResourceResolver.downloadMVSDataset(RequestId:wnfwb): Error renaming temporary download file see also screenshot above ) ...

Is there a way to trigger a reparse from our custom vscode extension ? We are now doing commands.executeCommand('zopeneditor.zapp.compiler.options.change');

(because we copy some copybooks from a server and that is after the cobol parser has parsed the file)

Thank you!

phaumer commented 1 month ago

@kraabrsg reviewing your log file I am not certain I follow what is happening. You are opening a COBOL file, but the I see requests such as languageClient/stop for language":"pl1" six times although no PL/I file was opened. Finally I see IBM Z Open Editor extension has been deactivated for a restart., which will trigger cleanup of any temp folders created so far, which probably causes the error. (You can use the setting zowe.files.temporaryDownloadsFolder.cleanup to prevent that.) Somehow though the COBOL language server process is still requesting files even though the editor is in shutdown mode. It seems that files that get downloaded might be deleted as part of that shutdown making the copybooks downloads random.

Why are you trying to shutdown our PL/I language server when you are editing COBOL programs? If you are editing Pl/I with a competitor product I recommend using a VS Code profile that does not have our extension installed and when editing COBOL use a different VS Code profile that uses our extension as is. (Or use my colleagues recommendation from the other issue of overriding the files associations so that our language server does not start.)

One more observation about the ZAPP file. I do not recommend pointing to copybook folders outside of the workspace in which the ZAPP file is located. All pathnames should be subfolders of the ZAPP file location. If the copybooks are in another workspace then there should be another ZAPP file in that workspace pointing to the subfolder there. Other users might be constructing their multi-root workspaces in different way storing the workspaces in different relative locations and then the ZAPP file path that has a relative path to parent folders will not work.

kraabrsg commented 1 month ago

Hi @phaumer thanks for getting back on this.

Why are you trying to shutdown our PL/I language server when you are editing COBOL programs?

Well we have our own made pli extension (e.g. because of preprocessor, where is a programm used, hover for plibuiltin keywords to provide description etc.,), not a competitor product. With several extension, and an installer via a command the user can choose what to install (cobol, or pli, or both, alongside other options). We choosed to used the file associations, or better said, we changed the id of our pli language implementation...

As for the ZAPP files we have it now working. We have two local folders in the zapp file. The first is a user syncronized folder from the mainframe(pds). The other is a somewhat global folder, where we do a quick parse of cobol copybooks and load them locally from a server. This avoids mainframe requests via zosmf. If we had to add mvs pds we need to tweak with your mentioned setting: zowe.files.temporaryDownloadsFolder.cleanup (thanks for that tip!)

Folder with workspacefile
  .vscode folder
      folder containing subfolder with user sync pds
      cobol copybook folder 1
   subfolder containing zapp file

zapp.json locations: "../.vscode/INC"  
etc...
phaumer commented 3 weeks ago

Closing this issue for now as the workaround is reassigning the file extensions to different language ids.