finos / perspective

A data visualization and analytics component, especially well-suited for large and/or streaming datasets.
https://perspective.finos.org/
Apache License 2.0
7.95k stars 1.06k forks source link

Fix 2.7.0 Migration #2444

Closed ada-x64 closed 7 months ago

ada-x64 commented 7 months ago

This PR ensures that the version created by the migration script matches the version in the package.json. It does so by inlining the package version and bumping the version as needed.

ada-x64 commented 7 months ago

@texodus I've updated this PR with some general cleanup of the migration scripts and I've replaced the copied file with an import from node_modules.

I think there is still some thinking to be done about how we're going to go about migrations in the future. The way I understand it, there are two ways to do API versioning. I'll call them from x and to x.

To-x migration scripts migrate a configuration from version x-1 to x. For example, 2-6-1.ts would migrate from 0.0.0 to 2.6.1.

From-x migration scripts migrate a configuration from version x to x+1. For example, script 2-6-1.ts would migrate from 2.6.1 to 2.7.0.

We're currently implementing the from-x migrations. The main issue with this approach, as we've discussed, is that there is no way of knowing what the next version will be. I've worked around this by simply placing the latest package version on top. In addition to fixing the latest-package version, the troubles we were having with versioning I think were caused by the mere fact that version migrations did not take the package version into account, which has been fixed in this PR.

There are other problems. Because we'll be developing API changes on master, and those migrations will be happening on the same version of perspective as the previous API version, we're still going to have to deal with a form of idempotency in the migrations. That is, we'll need to make sure that the current migration script can upgrade from any version x-n to version x, and stay at version x without change. (i.e., convert(convert({version: x-n})) == convert({version: x-n}) This shouldn't be too hard to accomplish, but is just something we'll need to keep in mind.