Allow to pass predefined field to type definitions

[christoph-buente]

In the process of generating the schema for shopify webhooks i came across a bunch of fields, that i know the format of, but the schema-guru fails to find out correctly. For example lat,lon coordinates, or currencies. My shop is in one currency only, so the guru will read it as string "EUR" instead of enum-set ISO 4217.

Is it feasable to ask for an option to provide a mapping JSON file like this:

    {
        "mappings":  {
            "currency": {
                "type" : "string",
                "enum" : [ "COU", "HRK", "USD", "CHF", "FKP", "CUP", "ANG", "AED", "MUR", "EGP", "MXN", "MRO", "RUB", "TMT", "UYU", "RON", "XDR", "KGS", "CUC", "CVE", "XPT", "LTL", "PLN", "CDF", "XAU", "TJS", "COP", "XXX", "MZN", "BAM", "ERN", "UGX", "BND", "TWD", "SYP", "VUV", "PAB", "LSL", "XBC", "LKR", "QAR", "NZD", "VEF", "LBP", "BZD", "GMD", "LYD", "GYD", "DZD", "IDR", "NGN", "SGD", "LAK", "KZT", "SZL", "GIP", "SBD", "BWP", "CAD", "RWF", "HUF", "TND", "ISK", "UAH", "MYR", "MKD", "XTS", "LVL", "BGN", "XPD", "XAF", "BTN", "SSP", "SDG", "BOB", "BRL", "AZN", "XBB", "MWK", "CNY", "BYR", "EUR", "GTQ", "PEN", "NOK", "SLL", "JPY", "KHR", "XBD", "KRW", "XOF", "AFN", "YER", "ETB", "DJF", "PHP", "ZMW", "XFU", "LRD", "TOP", "NIO", "HKD", "KPW", "SRD", "TTD", "PYG", "ARS", "HTG", "MDL", "JOD", "ZAR", "GNF", "NAD", "AWG", "MOP", "KES", "SAR", "FJD", "UZS", "SHP", "IQD", "ALL", "MNT", "MMK", "MVR", "CRC", "DOP", "PKR", "MAD", "JMD", "CLF", "XPF", "PGK", "MGA", "AOA", "CHE", "BOV", "WST", "VND", "THB", "SCR", "XBA", "IRR", "HNL", "AUD", "BDT", "STD", "GBP", "SEK", "BIF", "ILS", "BSD", "CZK", "XAG", "GEL", "INR", "MXV", "CLP", "RSD", "BHD", "TZS", "XCD", "BMD", "SOS", "OMR", "AMD", "DKK", "KYD", "UYI", "TRY", "KWD", "KMF", "BBD", "USN", "GHS", "NPR", "CHW", "USS" ]
            },
            "latitude": {
                "type": "number",
                "minimum": -90,
                "maximum": 90
            },
            "longitude": {
                "type": "number",
                "minimum": -180,
                "maximum": 180
            }
        }
    }

And whenever a field is found with this name, it will directly jump to the given mapping. Does that make sense?

[alexanderdean]

Interesting idea! @chuwy, @fblundun, thoughts?

[chuwy]

Yes, that makes sense and I like this idea. Property names often introduce more information than actual values. Do you think we need to provide any mappings by default, like with enum sets?

[chuwy]

@christoph-buente I assume that by min and max you meant minimum and maximum properties. To make it simple I think we should just try to put content of mapping straight into Schema, so it also would be better to put a full enum property rather than invented here enum-type. We don't want users to learn one more domain-specific language. Also, I assume that values still have a priority over mappings. If SG encounter an instance with name latitude and value -100 I think mapping should be discarded. Do you agree?

[christoph-buente]

Yes, i mean minimum and maximum. Changed it in the example. I also think it should be as simple as it can possibly be. I'm not sure how it would look like with a full enum. Like with currency with the full set available currencies? Or could it also be a regular expression?

And of course there should be at least a warning or an error, if the mapping doesn't match with the observed values. Or maybe even discarded.

[chuwy]

@christoph-buente could you also elaborate on your example with EUR? Do you use --enum-sets all option and it still not recognizes currency enum set or you omitted this option?

[christoph-buente]

I just found out about the --enum-sets all setting. And yes, it works.

[chuwy]

Cool! Feel free to add your suggestions: https://github.com/snowplow/schema-guru/issues/120