Explore changing the data editor to be jsonschema driven

[mike-hogan]

Change the Record Editor to be driven off of jsonschema definition instead of Models

This will require mapping Models to JSON Schema definitions, and mapping DataRecords back and forth between plain json.

JSON Schema supports regex validation statements, some number validations, etc, and supports extension. So there might be mileage in this, but it might be best regardless for errors to be passed down thru the component tree, and validation be done about the root UI element, so custom validation is possible.

Question is: why am I think about this? If the component uses something standard, it might be more generally useful, and folks might adopt it more, use it more, test it more, augment it more. It just “feels” better to play with open standards where possible.

If a jsonschema driven editor, with extensions like document generation etc, can deliver the UX we’re aiming for, then it makes sense to do that.

[mike-hogan]

Here is a json schema for the invoice test model we have been using. Created with the help of https://bjdash.github.io/JSON-Schema-Builder/

{
    "type": "object",
    "properties": {
        "invoiceNumber": {
            "type": "string",
            "description": "Invoice Number"
        },
        "description": {
            "type": "string",
            "description": "Description of invoice"
        },
        "lineItems": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "quantity": {
                        "type": "integer",
                        "minimum": 0
                    },
                    "item": {
                        "type": "string"
                    },
                    "price": {
                        "type": "number",
                        "minimum": 0,
                        "multipleOf": 0.01
                    }
                },
                "required": [
                    "quantity",
                    "item",
                    "price"
                ]
            }
        },
        "customer": {
            "type": "object",
            "properties": {
                "firstName": {
                    "type": "string"
                },
                "lastName": {
                    "type": "string"
                },
                "email": {
                    "type": "string",
                    "pattern": "^\\w+@[a-zA-Z_]+?\\.[a-zA-Z]{2,3}$"
                },
                "phone": {
                    "type": "string",
                    "pattern": "^[\\+]?[(]?[0-9]{3}[)]?[-\\s\\.]?[0-9]{3}[-\\s\\.]?[0-9]{4,6}$"
                },
                "address": {
                    "type": "object",
                    "properties": {
                        "line1": {
                            "type": "string"
                        },
                        "line2": {
                            "type": "string"
                        },
                        "postcode": {
                            "type": "string"
                        }
                    },
                    "required": [
                        "line1",
                        "postcode"
                    ]
                }
            },
            "required": [
                "firstName",
                "email",
                "phone",
                "address"
            ]
        }
    },
    "required": [
        "invoiceNumber",
        "customer"
    ]
}

[mike-hogan]

You can live test the schema using this React implementation of the same idea: https://rjsf-team.github.io/react-jsonschema-form/

[mike-hogan]

Here is a jsonschema for an invoice containing many customers, with each customer containing many addresses. Silly example, but good to exercise our code:

{
    "type": "object",
    "properties": {
        "invoiceNumber": {
            "type": "string",
            "description": "user name"
        },
        "customers": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "firstName": {
                        "type": "string"
                    },
                    "lastName": {
                        "type": "string"
                    },
                    "email": {
                        "type": "string",
                        "pattern": "^\\w+@[a-zA-Z_]+?\\.[a-zA-Z]{2,3}$"
                    },
                    "addresses": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "line1": {
                                    "type": "string"
                                },
                                "line2": {
                                    "type": "string"
                                },
                                "postcode": {
                                    "type": "string"
                                }
                            },
                            "required": [
                                "line1",
                                "postcode"
                            ]
                        }
                    }
                },
                "required": [
                    "firstName",
                    "email",
                    "addresses"
                ]
            }
        }
    },
    "required": [
        "invoiceNumber"
    ]
}

[mike-hogan]

Comments on current progress (which is awesome!)

• [x] Do one interesting things like formula cells. Maybe age of customer from date of birth
• [x] On save, make it async and handle error from that, which will have the same error structure i.e the backend might fail to save, and will say, for example "invoice.customer.email is not unique"
• [x] If Cell A is a currency and Cell B is a currency, and Cell C is a look up of the exchange rate between those two cells, when A or B change, then C will need to recalc. Which means it will need to look like it is "loading", and we should not be able to save until it resolves.
• [x] do the error validation at the "top" and pass down an error structure that is the same as the data structure- jsonschema is not enough
• [x] Array item containing an object that has a required field (many customers, each with one address, where the post code is mandatory)
• [x] Array of min length e.g. has to have three customers
• [x] Also max length
• [x] Each cell renderer is configured on the "outside" by type i.e string, number, date, image renderer

[Metehan-Altuntekin]

I have managed to set up a validation system with an errors store in this commit. The error object is the same structure as the record object and therefore each error should be passed to components recursively.

[Metehan-Altuntekin]

JSON Schema seems to not have a support for formula fields. However I could find a way of handling them with giving custom properties to such fields. Probably less than ideal.

[mike-hogan]

Next steps:

• [x] When I click save, I want to be able to issue SQL statements to store the changes. So I want to see `invoice.customer.email changed from X to Y, invoice.customer.address.postcode changed from A to B, invoice.lineItems[0] was deleted, invoice.lineItems[2] was added and its values are A,B,C"
• [x] Using these events, I want to be able to click undo, and undo the change the user made
• [x] I want this undo function to work for formulas, so if I change user.dateOfBirth, and user.age is calculated, I want to know that in the data user.dateOfBirth has changed and user.age has changed (so I can make the right SQL) but if I click undo, both values are undone
• [x] When a cell renderer can be registered on the "outside" i.e. I want to be able to say "from string types, use this svelte component", make an image cell type
• [x] The make a data range cell type, that stores its from and to dates in an object in the data structure, but uses one cell in the form to show the two dates and to edit the two dates
• [x] The buttons that say Add Nested Model should say "Add customer" or "Add address" etc
• [x] Prove that dependent formulas work, using the json schema I will post below.

[mike-hogan]

Here is the schema for the task above that proves that dependent formulas work:

{
    "type": "object",
    "properties": {
        "invoiceId": {
            "type": "string",
            "description": "user name"
        },
        "prices": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "item": {
                        "type": "string"
                    },
                    "price": {
                        "type": "number"
                    }
                },
                "required": [
                    "item",
                    "price"
                ]
            }
        },
        "items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "item": {
                        "type": "string"
                    },
                    "quantity": {
                        "type": "number"
                    },
                    "total": {
                        "type": "number"
                    }
                },
                "required": [
                    "item",
                    "quantity"
                ]
            }
        },
        "subtotal": {
            "type": "number"
        },
        "tenPercentSalesTax": {
            "type": "number"
        },
        "total": {
            "type": "number"
        }
    },
    "required": [
        "invoiceId"
    ]
}

And here is some sample data:

{
  "invoiceId": "invoice#22",
  "prices": [
    {
      "item": "apple",
      "price": 0.89
    },
    {
      "item": "banana",
      "price": 0.76
    }
  ],
  "items": [
    {
      "item": "apple",
      "quantity": 2
    },
    {
      "item": "banana",
      "quantity": 4
    }
  ]
}

What this means is that the cost for one apple is 0.89 and the cost for one banana is 0.76. This is meant to be reference data. The items array is the shopping cart. In contains 2 apples and 4 bananas. The total for the apple items line should be 2 x 0.89 and the total for the banana items line should be 4 x 0.76. If I change the cost of an apple or the cost of a banana then these should recalculate. if I change the quantity the associated line should recalculate. The subtotal if the sum of all the total of all the items. The tenPercentSalesTax is 10% of subtotal. Total is that added to the subtotal.

[Metehan-Altuntekin]

Cozemble specific properties like formula are moved into a coz object in the model. That way we will make it more compatible for non Cozemble use cases.

[mike-hogan]

New incarnation is #45