shuttle-hq / synth

The Declarative Data Generator
https://www.getsynth.com/
Apache License 2.0
1.37k stars 108 forks source link

schema specification in yaml #327

Closed gopik closed 2 years ago

gopik commented 2 years ago

Required Functionality Current implementation requires that the schemas be specified in json format. This is too verbose compared to yaml.

Proposed Solution A simplest fix could be add an option that first converts the provided yaml to json and uses the json as schema spec, with this requiring no change in the code impl. We could always do this in shell, but would be a useful addition if it's preferred by many.

gopik commented 2 years ago

We could just configure yaml deserializer for the config.

AlexMikhalev commented 2 years ago

I don't think it will be that simple: I tried to use toml and yaml in my other project and I didn't like end result - creating nested structures in yaml/toml is way more complex than in json. Below is a part of the sample JSON config. How would you turn it into yaml and what would be benefit?

            "type": "datasource",
            "path": "json:iodata.json",
            "cycle": true
          },
            "timestamp": {
                "type": "date_time",
                "format": "%Y-%m-%dT%H:%M:%S%z",
                "begin": "2000-01-01T00:00:00+0000",
                "end": "2020-01-01T00:00:00+0000"
              },
              "device_id": {
                "type": "same_as",
                "ref": "device.content.id"
              },
              "device_type": {
                "type": "same_as",
                "ref": "device.content.device_type"
              }
    }
}
iamwacko commented 2 years ago

While I don't see much of a benefit of using YAML, nesting in YAML isn't that complicated. To turn what you have into YAML. the braces, commas, and some of the parentheses are removed.

            type: "datasource"
            path: "json:iodata.json"
            cycle: true
            timestamp: 
                type: "date_time"
                format: "%Y-%m-%dT%H:%M:%S%z"
                begin: "2000-01-01T00:00:00+0000"
                end: "2020-01-01T00:00:00+0000"
            device_id: 
                type: "same_as"
                ref: "device.content.id"
            device_type: 
                type: "same_as"
                ref: "device.content.device_type"
AlexMikhalev commented 2 years ago

Then we need to turn path: "json:iodata.json" too. Let's stick to JSON for now. I am going to close the ticket until we both will see the benefit of adding YAML.