ZE API: cannot append properties to a custom profile with user-specific config file

roman-kupriyanov commented 2 years ago

Describe the bug

I have created an additional zowe.config.user.json file to accompany my zowe.config.json with existing profiles (see examples in the Additional context section). The idea is to append some user specific fields to the existing profile from global team config using the user-specific config.

But on the initialization of Zowe Explorer API I keep getting the error:

'ProfInfoErr: Failed to find property rejectUnauthorized in the profile base_def'

(to be precise: during the profiles reloading ZoweVsCodeExtension.getZoweExplorerApi().getExplorerExtenderApi().reloadProfiles())

In the same time Zowe CLI seems to work good with this setup.

The only workaround I found so far is to duplicate the base_defprofile in the user-specific config, in this case it is possible to pass the profiles reloading and to use the merged profile successfully.

Expected behavior

ZE API should allow to retrieve the profiles merged from global and user configs.

Desktop (please complete the following information):

OS: MacOS
Zowe Explorer Version: 2.2.0
(Optional) Zowe CLI Version: 7.4.0
(Optional) Are you using Secure Credential Store? Yes

Additional context

We have a zowe.config.json with some custom profiles:

{
    "$schema": "./zowe.schema.json",
    "profiles": {
        "base_def": {
            "type": "base",
            "properties": {
                "rejectUnauthorized": false
            },
            "secure": [
                "user",
                "password"
            ]
        },
        "test": {
            "type": "endevor",
            "properties": {
                "host": "test.com",
                "port": 1234,
                "protocol": "http",
                "basePath": "/"
            },
            "secure": []
        },
        "test-location": {
            "type": "endevor-location",
            "properties": {
                "instance": "INST",
                "environment": "ENV",
                "system": "SYS",
                "subsystem": "SBS",
                "stageNumber": "1",
                "type": "*"
            },
            "secure": []
        },
        "test-zosmf": {
            "type": "zosmf",
            "properties": {
                "host": "1.2.3.4",
                "port": 43,
                "protocol": "https"
            },
            "secure": [
                "user",
                "password"
            ]
        },
    },
    "defaults": {
        "base": "base_def",
        "endevor": "test",
        "endevor-location": "test-location",
        "zosmf": "test-zosmf"
    },
    "autoStore": true
}

And a zowe.config.user.json:

{
    "$schema": "./zowe.schema.json",
    "profiles": {
        "test-location": {
            "type": "endevor-location",
            "properties": {
                "ccid": "develop1",
                "comment": "Changes by develop1"
            },
            "secure": []
        }
    },
    "defaults": {},
    "autoStore": true
}

Both are in .zowe directory.

Zowe CLI output:

❯ zowe config ls
profiles: 
  base_def: 
    type:       base
    properties: 
      rejectUnauthorized: false
      user:               (secure value)
      password:           (secure value)
    secure: 
      - user
      - password
  test: 
    type:       endevor
    properties: 
      host:     test.com
      port:     1234
      protocol: http
      basePath: /
    secure: 
      (empty array)
  test-location: 
    type:       endevor-location
    properties: 
      instance:    INST
      environment: ENV
      system:      SYS
      subsystem:   SBS
      stageNumber: 1
      type:        *
      ccid:        develop1
      comment:     Changes by develop1
    secure: 
      (empty array)
  test-zosmf: 
    type:       zosmf
    properties: 
      host:        1.2.3.4
      port:        43
      protocol:    https
    secure: 
      - user
      - password
defaults: 
  base:             base_def
  endevor:          test
  endevor-location: test-location
  zosmf:            test-zosmf
autoStore: true

github-actions[bot] commented 2 years ago

Thank you for creating a bug report. If you haven't already, please ensure you have provided steps to reproduce it and as much context as possible.

roman-kupriyanov commented 2 years ago

Not sure if it would be somehow helpful in this case.. I have just checked it out that it works if team config and user config do not have the "base" type profile specified at all.

zFernand0 commented 2 years ago

This sounds related to zowe/imperative#881

@roman-kupriyanov, once ZE v2.3 is out, could you confirm if the issue still persists?

roman-kupriyanov commented 2 years ago

@zFernand0 Sure, I will recheck it and let you know, thanks!

roman-kupriyanov commented 1 year ago

Hi @zFernand0

Good news, I have tested the latest ZE v2.3.0 with our extension and team/user config setup, described above, and now it is initialized properly without errors, yay! Thank you for the help 😃

I have noticed some kind of a related issue for ZE though. Now it does not respect a base profile specified in the User config on the global level (in ~/.zowe folder), but it seems that it respects it on the project level (if an opened VSCode workspace folder includes Team and User configs). In the same time I tried the same setup with Zowe CLI v7.6.2 and it work fine with the global level User configs too.

I must say I do not know where exactly it is expected for User config to be located (probably, most of the time on the project level, not on the global one), but since Zowe CLI support both scenarios, I would expect ZE to do the same. And we actually have some customers who might be interested to specify their base profile in User config, not to be shareable with anyone else.

Examples:

~/.zowe/zowe.config.json config:

{
    "$schema": "./zowe.schema.json",
    "profiles": {
        "test": {
            "type": "endevor",
            "properties": {
                "host": "test.com",
                "port": 1234,
                "protocol": "http",
                "basePath": "/"
            },
            "secure": []
        },
        "test-location": {
            "type": "endevor-location",
            "properties": {
                "instance": "INST",
                "environment": "ENV",
                "system": "SYS",
                "subsystem": "SBS",
                "stageNumber": "1",
                "type": "*"
            },
            "secure": []
        },
        "test-zosmf": {
            "type": "zosmf",
            "properties": {
                "host": "1.2.3.4",
                "port": 43,
                "protocol": "https"
            },
            "secure": [
                "user",
                "password"
            ]
        },
    },
    "defaults": {
        "endevor": "test",
        "endevor-location": "test-location",
        "zosmf": "test-zosmf"
    },
    "autoStore": true
}

And ~/.zowe/zowe.config.user.json config:

{
    "$schema": "./zowe.schema.json",
    "profiles": {
        "base_def": {
            "type": "base",
            "properties": {
                "rejectUnauthorized": false
            },
            "secure": [
                "user",
                "password"
            ]
        },
        "test-location": {
            "type": "endevor-location",
            "properties": {
                "ccid": "develop1",
                "comment": "Changes by develop1"
            },
            "secure": []
        }
    },
    "defaults": {
        "base": "base_def"
    },
    "autoStore": true
}

Zowe CLI output:

❯ zowe config ls
profiles: 
  test: 
    type:       endevor
    properties: 
      host:     test.com
      port:     1234
      protocol: http
      basePath: /
    secure: 
      (empty array)
  test-location: 
    type:       endevor-location
    properties: 
      instance:    INST
      environment: ENV
      system:      SYS
      subsystem:   SBS
      stageNumber: 1
      type:        *
      ccid:        develop1
      comment:     Changes by develop1
    secure: 
      (empty array)
  test-zosmf: 
    type:       zosmf
    properties: 
      host:        1.2.3.4
      port:        43
      protocol:    https
    secure: 
      - user
      - password
  base_def: 
    type:       base
    properties: 
      rejectUnauthorized: false
      user:               (secure value)
      password:           (secure value)
    secure: 
      - user
      - password
defaults: 
  base:             base_def
  endevor:          test
  endevor-location: test-location
  zosmf:            test-zosmf
autoStore: true

I have even tried this weird flow and it worked:

Set the zowe.config.json and zowe.config.user.json configs in ~/.zowe folder.
Open any folder without configs in VSCode (or do not open at all): the extension have no information about credentials specified (the base profile is not merged it seems).
Open ~/.zowe folder in VSCode (with Open Folder dialog): and now it detects the base profile information.

Because of this scenario I have an assumption about that it works only with the project level configs and not global ones. But maybe something else is involved. In the same time Zowe CLI works in all cases, whether you are in the ~/.zowe folder or in any else.

zFernand0 commented 1 year ago

After some discussion, the CLI squad has taken time to document how the different layers (configuration files) are merged. For details, see How_config_files_are_merged

Please feel free to re-open the issue if there is something missing from the doc 😋

roman-kupriyanov commented 1 year ago

Hi @zFernand0,

I retested on the latest ZE version (v2.6.1) and unfortunately it is still the case for base profile merging (https://github.com/zowe/vscode-extension-for-zowe/issues/1895#issuecomment-1266984518). I am able to use my secured name and password from base profile, but only if it is specified in team config or in the project level config.

For details, see How_config_files_are_merged

I did not find any mentions or restrictions related exactly to this case in that document or maybe you can help me to understand if I am missing something. And by the way, it is a very nice document which really does a good job in explaining the merging mechanism in details. Thanks for sharing!

I am not sure that I should reopen the issue for exactly ZE since it might be more common Zowe problem. Let me know if you think there is more suitable place for it.

zowe / zowe-explorer-vscode

ZE API: cannot append properties to a custom profile with user-specific config file #1895