IBM / zopeneditor-about

IBM Z Open Editor: File issues here!
https://ibm.github.io/zopeneditor-about
Apache License 2.0
50 stars 20 forks source link

Zowe Explorer 2.2.0 - Unable to create credentials when searching for remote COPYBOOKs in a new Workspace with zowe.config.json inside #260

Closed FALLAI-Denis closed 2 years ago

FALLAI-Denis commented 2 years ago

Development environment used

Problem Description

Since upgrading to Zowe Explorer 2.2.0, creating credentials for the Zowe CLI (V2) profile no longer works.

Detailed steps for reproducing the problem:

  1. Create a new Workspace
  2. Create a zowe.config.json at root folder of Workspace, with same lpar as profile used by Z Open Editor
  3. Create a new COBOL file
  4. Open Z Open Editor on this COBOL file and add reference to COPYBOOK (or copy/ paste an existing COBOL file with COPYBOOKs)
  5. Z Open Editor attempts to access z/OS to find remote COPYBOOKs (zapp.yaml well configured)
  6. Z Open Editor asks for the user / password of the z/OS account
  7. Z Open Editor fails in accessing COPYBOOKs
  8. Z Open Editor no longer attempts to access any remote COPYBOOKs

Remarks :

Observed behavior

No user account revokation! Hura!

run command "zowe files list ds ...." in VS Code Terminal: ask for credentials, and OK after for Z Open Editor to access to remote mainframe. So problem is in Z Open Editor 2.1.0 + Zowe Explorer 2.2.0.

FALLAI-Denis commented 2 years ago

PS :

And unable to update an existing contextual credentials (already created credentials with zowe.config.json in root folder)...

And revocation of remote user account on z/OS... 👎

phaumer commented 2 years ago

Fixed with 2.1.1 and Zowe Explorer 2.2.0.

FALLAI-Denis commented 2 years ago

@phaumer

Sorry, but the problem is still present in version 2.1.1 of Z Open Editor... Please reopen this issue.

JillieBeanSim commented 2 years ago

@FALLAI-Denis working to recreate issue and noticed weird behavior if my environment wasn't just right. So I have a couple questions for you if you don't mind.

  1. Is Zowe CLI installed and if so what version
  2. Are you storing credentials securely using Zowe CLI 7.x and not SCS plugin with CLI v6.x?
  3. Do you have a global config in place as well as project level and if so and storing securely, are the secure arrays in the same location on both? ie. base profile
  4. Is this a multi-root workspace, single, or both are you seeing the issue on?
  5. Are you using default Zowe Home (~/.zowe) or the environment variable ZOWE_CLI_HOME to set the home location?
FALLAI-Denis commented 2 years ago

Hi @JillieBeanSim

  1. Is Zowe CLI installed and if so what version --> Yes, @zowe/cli 7.1.2 --> @broadcom/endevor-for-zowe-cli 7.1.0 --> @zowe/zos-ftp-for-zowe-cli 2.0.0 --> use of nvm to switch node.js versions (1 node.js version for Zowe CLI V1, 1 node.js version for Zowe CLI V2, currently Zowe CLI V2, switch of .zowe folder in User folder synchronised with switch of node.js version) --> node.js v12.13.0 --> npm 6.12.0 --> Workstation OS: Windows 10 20H2
C:\Users\xxxxxxxx>where zowe
C:\Users\xxxxxxxx\zowe                <--- ???
C:\Program Files\nodejs\zowe
C:\Program Files\nodejs\zowe.cmd
C:\Users\xxxxxxxx\.zowe\bin\zowe.exe

C:\Users\xxxxxxxx>dir zowe
 Le volume dans le lecteur C s’appelle VOL_1
 Le numéro de série du volume est 9810-1EE0
 Répertoire de C:\Users\xxxxxxxx
24/04/2022  13:28                 0 zowe
               1 fichier(s)                0 octets
               0 Rép(s)  190 315 368 448 octets libres
C:\Users\xxxxxxxx>type zowe
C:\Users\xxxxxxxx>

C:\Users\xxxxxxxx>where npm
C:\Program Files\nodejs\npm
C:\Program Files\nodejs\npm.cmd

C:\Users\xxxxxxxx>where node
C:\Program Files\nodejs\node.exe

C:\Users\xxxxxxxx\.zowe>dir
 Le volume dans le lecteur C s’appelle VOL_1
 Le numéro de série du volume est 9810-1EE0
 Répertoire de C:\Users\xxxxxxxx\.zowe
28/06/2022  18:27    <DIR>          .
28/06/2022  18:27    <DIR>          ..
28/06/2022  18:27    <DIR>          .zosexplorer    <-- ???
04/05/2022  18:54    <DIR>          bin
10/04/2020  19:54    <DIR>          imperative
23/05/2022  19:02    <DIR>          plugins
09/06/2022  18:49    <DIR>          profiles          <-- automatic creation
02/07/2020  19:16    <DIR>          profiles_old      <-- manually renamed
12/06/2020  00:20    <DIR>          profile_links
10/04/2020  19:54    <DIR>          settings
21/07/2022  11:56    <DIR>          web-help
10/04/2020  19:54    <DIR>          zowe
14/06/2022  12:12             1 500 zowe.config-pour-jenkins.json
13/07/2022  18:00             7 823 zowe.config.json
24/05/2022  20:31               835 zowe.config.user-modele.json
21/05/2022  11:04               795 zowe.config.user.json
14/06/2022  12:09            24 549 zowe.schema-pour-jenkins.json
23/05/2022  19:00            46 423 zowe.schema.json
               6 fichier(s)           81 925 octets
              12 Rép(s)  190 314 541 056 octets libres
  1. Are you storing credentials securely using Zowe CLI 7.x and not SCS plugin with CLI v6.x? --> Yes for Zowe CLI 7.x and Yes for Zowe CLI 6.x

  2. Do you have a global config in place as well as project level and if so and storing securely, are the secure arrays in the same location on both? ie. base profile --> Yes for global (User Home Directory), and Yes for project level (at root of VS Code Workspace) --> I don't anderstand "are the secure arrays in the same location on both? ie. base profile"... I don't choose location for secure arrays? --> no "base" profile --> profile names are same, in fact, config.user.json in global config place is a superset of config.user.json in project level

C:\a_workspace>dir
 Le volume dans le lecteur C s’appelle VOL_1
 Le numéro de série du volume est 9810-1EE0

 Répertoire de C:\a_workspace

08/08/2022  17:18    <DIR>          .
08/08/2022  17:18    <DIR>          ..
08/08/2022  18:19    <DIR>          .build
05/08/2022  21:27             1 305 .gitattributes
05/08/2022  21:27               244 .gitignore
05/08/2022  21:27    <DIR>          .imports
08/08/2022  18:11    <DIR>          .reports
05/08/2022  21:27    <DIR>          .sonar
05/08/2022  21:27    <DIR>          .vscode
05/08/2022  21:27             1 131 .zosattributes
05/08/2022  21:27             1 263 application.properties
05/08/2022  21:27    <DIR>          dist
05/08/2022  21:27               526 Jenkinsfile
05/08/2022  21:27             1 688 readme.md
05/08/2022  21:27               675 sonar-project.properties
05/08/2022  21:27    <DIR>          src
08/08/2022  17:18             1 205 zapp.yaml
05/08/2022  22:36             4 096 zowe.config.json
05/08/2022  21:27               262 zowe.config.user.json.sample-copier-pas-renommer
05/08/2022  21:27            47 207 zowe.schema.json
              11 fichier(s)           59 602 octets
               9 Rép(s)  190 134 829 056 octets libres
  1. Is this a multi-root workspace, single, or both are you seeing the issue on? --> No, single root workspace, but sometime multiple VS Code instances active at the same time, but each being mono Workspace.
JillieBeanSim commented 2 years ago

Thanks @FALLAI-Denis. I have never tried having both v1 & v2 installed at the same time. v2 will require node 14 for the CLI and VS Code components. I was just testing if nvm could keep 6.x and 7.x separate by changing from node 14 to node 12 for 6.x then installed @zowe/cli@zowe-v1-lts and the first error I get is about the credential manager for 7.x in place

[2022/08/08 12:47:40.464] [ERROR] [CredentialManagerFactory.js:148] Failed to override the credential manager with one provided by "@zowe/cli"
[2022/08/08 12:47:40.470] [ERROR] [CredentialManagerFactory.js:150] You attempted to override the "CredentialManager" setting with a plugin named "@zowe/cli" that is not installed and loadable.
We will use a "CredentialManager" that purposely fails until you reconfigure.

I haven't even tried installing the SCS for 6.x yet. With having different credential managers between the 2 versions could be the root cause for much of the credential issues seen. I will continue looking into this, just wanted to update with stuff seen so far in recreating that environment.

JillieBeanSim commented 2 years ago

With some testing and following the below steps during changing versions this is what I found:

The file ~/.zowe/settings/imperative.json holds the credential manager. With 6.x & SCS this value is "CredentialManager": "@zowe/secure-credential-store-for-zowe-cli" With 7.x the value should be "CredentialManager": "@zowe/cli" The part to remember is when switching, before changing node from 6.x (12) to 7.x (14) is to run zowe scs revert -f then zowe config reset CredentialManager. The last command will change the file above to "CredentialManager": false and will need to be updated to the 7.x value to continue storing securely. When changing from 7.x to 6.x update the file above to the 6.x value and run command zowe scs update to change credential manager and update the stored values to the new manager.

Even with following these steps and working in only the CLI, when I go from 6.x to 7.x I continuously get prompted for credentials at each command like it's not storing it by the 7x credential manager. I could see this being an issue in the VS Code space as well.

nvm may keep the zowe installations separate but 1) the SCS plugin is showing in both node instances when running zowe plugins list and 2) the files created by the package are shared among the 2 which causes issues 😒

FALLAI-Denis commented 2 years ago

Hi @JillieBeanSim

I use ~/.zowe folders per Zowe CLI version, (I rename ~/.zowe folder if I want to switch from V1 to V2).

For Zowe CLI V2:

image

image

For Zowe CLI V1:

image

I have a Zowe CLI V1 version with node.js 14.18.0, and a Zowe CLI V2 version with node.js 12.13.0.

I have done various manipulations in the past:

I can try to uninstall everything at Zowe CLI V1 and V2 level and reinstall everything...

Is secure element storage shared between Zowe CLI V1 and Zowe CLI V2? How can I view / query the secure elements?

FALLAI-Denis commented 2 years ago

PS : something strange with nodejs... twice in PATH and not same location

image

C:\ProgramData\nodejs does'nt exists: image

C:\Program Files\nodejs is a symlink to C:\ProgramData\nvm\v12.13.0 wich contains node.js image

image

image

image

image

image

FALLAI-Denis commented 2 years ago

v2 will require node 14 for the CLI and VS Code components.

ah ah ! I use node v12.13.0 for Zowe CLI V2... is this the problem?

Node version not verified at install or at runtime for Zowe CLI?

Zowe CLI V2 works fine for me with node.js V12... problems concerns only use of Zowe (SDK) from VS Code.

JillieBeanSim commented 2 years ago

Hey @FALLAI-Denis you can find the secure Zowe elements in the Windows Credential Manager. The Zowe v2 VS Code components require node 14 or higher since node 12 went out of support right before the 2.0 release, I am not for certain if the Zowe CLI team did the same.

phaumer commented 2 years ago

The goal for the Zowe CLI and Explorer teams was not to support switching between versions, but rather have CLI v7 and Explorer 2 binaries work with v1 profiles as well as v2 profiles. As Billie mentioned switching has various risks as libraries are in different places, including the .zowe folder and then there are the issues with keytar being dependent on the node version as well as the format changes of the data stored. If I would personally attempt this using two different installations I would perhaps try running two separate Docker containers or even VMs on my laptop to ensure the proper isolation avoid side effects when switching, but we are not testing and supporting that either at the moment.

FALLAI-Denis commented 2 years ago

Hi @phaumer

The Zowe CLI version switch was to diagnose problems with IBM User Build working with IBM Dependency Based Build: works with Zowe CLI V1 and does not work with Zowe CLI V2.

FALLAI-Denis commented 2 years ago

@JillieBeanSim

Hi,

I noticed one thing:

FALLAI-Denis commented 2 years ago

Hi @phaumer, @JillieBeanSim

The problem seems to have disappeared.

Three actions were done, I don't know which solved the problem:

FALLAI-Denis commented 2 years ago

The problem no longer occurs.

phaumer commented 2 years ago

That is great news @FALLAI-Denis. Glad to hear it. Yes, there were many variables that had to tested against. Thanks a lot to @JillieBeanSim who did an excellent job of trying all the different combinations and fixing wherever she could.