[BUG] Problem when importing new system over CLI import.yml

[mordax7]

If you try to import new systems to SHIELD over the SHIELD CLI there is an error in the UI when clicking on a system. Error in the UI:

Uh-oh — Something broke.
An unknown error has occurred.

Error in the logs:

2019-12-04 23:31:10.187870234 +000 UTC /var/vcap/packages/shield/bin/shieldd: ERROR: GET /v2/tenants/8a13b9a1-6ccc-4496-b38f-e3c5388aa80e/systems/d0e94fba-904a-4188-9c1a-479bf7cbd9a5/config errored: unable to retrieve target configuration: agent metadata not found in database.

The import.yml file:

token: DISCARD
global:
  storage:
    - name: S3 Cloud Storage
      summary: |
        Public S3 cloud storage for all SHIELD tenants to use
      agent:  DISCARD:5444
      plugin: s3
      config: {
        access_key_id: "DISCARD",
        secret_access_key: "DISCARD",
        bucket: "DISCARD",
        s3_host: "DISCARD",
        region: "eu-central-1"
      }
  policies:
    - name: Long-Term Storage
      days: 90

users:
  - name:     DISCARD
    username: DISCARD
    password: DISCARD
    sysrole:  admin
    tenants:
      - name: Test tenants
        role: admin

tenants:
  - name: Test tenants mongo
    members:
      - user: DISCARD@local
        role: admin

    systems:
      - name:  MONGO IMPORT TEST
        agent:  DISCARD:5444 <- IP instead of DNS entry
        plugin: mongo
        config: {
          mongo_host: "DISCARD",
          mongo_port: "27019",
          mongo_user: "DISCARD",
          mongo_password: "DISCARD",
          mongo_database: "DISCARD",
          mongo_options: "--authenticationDatabase DISCARD",
      mongo_bindir: "/var/vcap/packages/shield-addon-mongo-tools-3.6/bin"
        }
        jobs:
          - name:    Monthly
            when:    daily 4am
            policy:  Long-Term Storage
            storage: S3 Cloud Storage
            retain:  1d

We found out that the problem gets fixed if you use an IP instead of a DNS entry for the agent value in the import.yml(I wrote it in the example). This does not work for us since the agents IP changes constantly and we would like to use the DNS entry to automate our backup procedure. Interesting is that the jobs are working normally if you trigger them over the CLI, just the UI view seems to have problems.

To Reproduce Steps to reproduce the behavior:

1. Write a import.yml to import the configuration into SHIELD
2. In the system section, when you are configuring the agent where you want to execute your backups on, use a URL instead of IP
3. Import the import.yml with the SHIELD CLI
4. Try opening the new system in the UI

Expected behavior You should see the overview of that systems configuration

Screenshots

SHIELD versions (please complete the following information):

• SHIELD Core: v8.6.0
• SHIELD CLI: v8.6.0

Browser version(s) (please complete if reporting a web UI bug):

• OS: Linux
• Browser: firefox, chromium
• Version: latest stable

[jhunt]

This is due to the fact that the web UI attempts to resolve plugin metadata based on the agents registered address, which is determined from the TCP tuple of the connecting registration. This is incidentally why we cannot abide by agents who are not mutually visible to the SHIELD core on the network (i.e. behind a NAT). We have designs to replace the current SSH fabric that SHIELD uses to communicate with agents with an Active Fabric, in which agents SSH into the SHIELD core and it doesn't matter what their address is or what network translation occurs in between: https://github.com/shieldproject/notebook/blob/master/proposals/active-ssh-fabric.md

Until that gets implemented. at some point in either the 8.7 or 8.8 releases, I don't expect the use case of DNS-names for agents to work outside of the CLI.

[jhunt]

Active SSH Fabric will be included in the new v9 SHIELD, as discussed on our May 28th Community Roadmap Call.