jonasvinther / medusa

A cli tool for importing and exporting Hashicorp Vault secrets
MIT License
475 stars 62 forks source link

medusa import: Unable to import empty secret files #58

Open thatInfrastructureGuy opened 3 years ago

thatInfrastructureGuy commented 3 years ago

When transferring secrets from one instance to another, I run medusa export which gives me structure similar to this:

A:
  B:
    key1: value1
  C: {}

Here A/C is empty secret.

When I run medusa import, only A/B is imported.

A:
  B:
    key1: value1

Secret A/C is not imported.

umutkacar commented 10 months ago

In my case, with medusa v0.6.0, I don't even see the empty keys on the export. Here is the source structure that I'm trying to migrate to a new Vault, for example:

.
└── secrets/data_science/
    ├── data_engineering/
    │   ├── debezium-backend/
    │   │   ├── global
    │   │   ├── production/
    │   │   │   └── global
    │   │   │       ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
    │   │   │       └── :JDBC_LOG_DWH_READONLY_PASSWORD
    │   │   └── staging/
    │   │       └── global
    │   │           ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
    │   │           └── :JDBC_LOG_DWH_READONLY_PASSWORD
    │   └── global
    └── global

Here is also a paths output for a better understanding, maybe:

secrets/data_science/data_engineering/debezium-backend/global
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/production/global:JDBC_LOG_DWH_READONLY_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_DEBEZIUM_PASSWORD
secrets/data_science/data_engineering/debezium-backend/staging/global:JDBC_LOG_DWH_READONLY_PASSWORD
secrets/data_science/data_engineering/global
secrets/data_science/global

But the export file only has:

data_engineering:
  debezium-backend:
    production:
      global:
        JDBC_LOG_DWH_DEBEZIUM_PASSWORD: omitted-s3cr3₺
        JDBC_LOG_DWH_READONLY_PASSWORD: omitted-s3cr3₺
    staging:
      global:
        JDBC_LOG_DWH_DEBEZIUM_PASSWORD: omitted-s3cr3₺
        JDBC_LOG_DWH_READONLY_PASSWORD: omitted-s3cr3₺

It's also the same with the json export:

{
  "data_engineering": {
    "debezium-backend": {
      "production": {
        "global": {
          "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
          "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
        }
      },
      "staging": {
        "global": {
          "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
          "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
        }
      }
    }
  }
}

There is another tool called safe which exports these empty keys indeed, by fails to import them.

Even if I get the intact json export from safe and try to import it with medusa, it does not create these empty keys. (I know the path structure of these two exports are different.)

{
  "secrets/data_science/data_engineering/debezium-backend/global": {},
  "secrets/data_science/data_engineering/debezium-backend/production/global": {
    "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
    "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
  },
  "secrets/data_science/data_engineering/debezium-backend/staging/global": {
    "JDBC_LOG_DWH_DEBEZIUM_PASSWORD": "omitted-s3cr3₺",
    "JDBC_LOG_DWH_READONLY_PASSWORD": "omitted-s3cr3₺"
  },
  "secrets/data_science/data_engineering/global": {},
  "secrets/data_science/global": {}
}

So the final result at the destination Vault, does not resemble the data structure of the source:

.
└── secrets/data_science/
    └── data_engineering/
        └── debezium-backend/
            ├── production/
            │   └── global
            │       ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
            │       └── :JDBC_LOG_DWH_READONLY_PASSWORD
            └── staging/
                └── global
                    ├── :JDBC_LOG_DWH_DEBEZIUM_PASSWORD
                    └── :JDBC_LOG_DWH_READONLY_PASSWORD

I'd really like to have the option to include empty keys on with import function.