If your program only handles data that disappears when it crashes or dies, it's a toy. 99% of programming languages today can only make toys. So people use a separate "database" for non-toys. But databases come with terrible programming languages. Maybe exception: Erlang, a real language with which you can naturally build long-running databases resilient to crashes; though it's most constraining. Also, Erlang has no type system.
Now, in any non-toy program with long-running data, a data schema upgrade is an eventual necessity, to deal with bitrot: tomorrow's needs are not yesterday's, and the program must evolve, with it the data. But all existing systems, including Erlang, make data schema upgrade a very difficult, traumatic endeavor, that can damage the system.
Can a typed language support long-running data AND schema upgrade, in a way that leads to correct-by-construction solutions?
That means that not just the "current state" of the program, but its deployment history, are taken into account by the system. Not just a variable's current type, but its past types, matter, with incrementally runnable transformations from the past to the future. Sometimes, the transformation is not localized to a variable, but to a group of variable. Sometimes, the upgrade has to be done in many phases to remain incremental, to build the necessary intermediate data structures and indexes.
If your program only handles data that disappears when it crashes or dies, it's a toy. 99% of programming languages today can only make toys. So people use a separate "database" for non-toys. But databases come with terrible programming languages. Maybe exception: Erlang, a real language with which you can naturally build long-running databases resilient to crashes; though it's most constraining. Also, Erlang has no type system.
Now, in any non-toy program with long-running data, a data schema upgrade is an eventual necessity, to deal with bitrot: tomorrow's needs are not yesterday's, and the program must evolve, with it the data. But all existing systems, including Erlang, make data schema upgrade a very difficult, traumatic endeavor, that can damage the system.
Can a typed language support long-running data AND schema upgrade, in a way that leads to correct-by-construction solutions?
That means that not just the "current state" of the program, but its deployment history, are taken into account by the system. Not just a variable's current type, but its past types, matter, with incrementally runnable transformations from the past to the future. Sometimes, the transformation is not localized to a variable, but to a group of variable. Sometimes, the upgrade has to be done in many phases to remain incremental, to build the necessary intermediate data structures and indexes.
TODO: update the description of this issue.