Create Changelog and migration path for the rewrite

This ticket's expected outcomes are a list of changes (primarily to identify the "breaking" ones), and a set of instructions on how to update existing Arranger setups using the new version.

For reference, the bulk of changes made thus far can be simplified as follows:

Architectural changes

The Arranger family now comprises only 3 modules: Server, Components, and Admin. In contrast to the previous 7 pieces, this architecture makes Arranger much easier to reason with both for maintainers and integrators.
As a result, Arranger Server is now a single package.
Admin is currently "out of service" while focusing on improving Server and Components.

Operational changes

All Arranger packages are now published to the Overture stack NPM scope.
We recommend using the latest NodeJS+NPM versions, and the packages may not be compatible with older, unsupported versions (<v14).
We removed the concept of "projects", as an Arranger setup was initially made to allow serving multiple indexes with individual configurations in a single service, which can be better done by running multiple Servers with their own configs.
Furthermore, server/component configuration is now done through files (or JSON objects, possibly), rather than being hosted in Elasticsearch as a "projects" index.
Those configs used to then be included in the GraphQL requests as *State, and while still available, now will be nested inside a configs parameter and grouped by Component family (e.g. configs > table > columns). We will document the new schema more explicitly (and visually?) once the changes are finished.
The Elasticsearch mapping is only made available in the Server responses when the enableAdmin flag is present/active. This was done for backwards compatibility reasons (e.g. some Aggregation components still rely on the mapping), but ideally we want to remove the mapping from most requests.
(WIP) The Server and Components packages are now TypeScript compatible, providing type and interface descriptions for their functionality. Still undergoing migration from Flow. The Type definitions will eventually be abstracted into a shared @types package, that all Arranger packages can reuse.
(WIP) Aside from prettier, both components and server also include updated eslint configs, to facilitate future implementation/refactoring. The rules and plugins are working, but may need to be adjusted as we go to better fit our needs/preferences.
(WIP) We now add .vscode settings, seeking to assist new contributors and maintainers in setting up their dev environment.

Functional changes

Arranger Server provides both health and liveness endpoints, useful for containerisation.
Arranger Components Table has a new max rows default of 100 for TSV downloads. This value can be customised as exporter props, pending on the ALLOW_CUSTOM_MAX_DOWNLOAD_ROWS Server env var being enabled. Server can also change the default max rows through MAX_DOWNLOAD_ROWS. If 0 is given as props from the UI side, the server will default to whatever value is set in that last env var. If that one is defined as 0, then no limit will be applied at all (which we strongly advice against in large datasets.)
(WIP) Components will now rely on Data Context + Theme providers, and their respective helper hooks. The implementation of the Theme itself, and its usage by the actual components is still not quite finished, but will allow both extensive style and functionality customisation.
The new Table component family, apart from sharing a state Context of their own, are now also significantly easier to compose and reason with. e.g. You can make your own tool bar, and (re)use it anywhere.

WIP List of breaking changes:

Projects do not exist anymore. In order to set or change the settings for Arranger services, an integrator will have to modify the configs directly (as a file or group of them). There's also a new GraphQL query hasValidConfig, to verify whether Arranger Server is correctly configured. More on this to be documented.
When trying to run Arranger Server with NodeJS versions older than v14, integrators will get error messages. This may require infrastructure updates for them (which is strongly encouraged anyway)
Arranger Components will not work anymore in versions older than 16.8 (because of Hooks). Though they should work correctly regardless of React version after that one, it is possible (though unlikely) some bugs may arise in React versions earlier than v17, but testing needs be done to assess this point. No "fixes" necessary in this case, as it is a "supported version" kind of deal.
While we fully migrate Arranger from Flow to TypeScript, it is possible (though unlikely) newly added type assertions could generate development/implementation errors in existing TypeScript apps. This is because we may have formerly allowed unintended usages, that are now being avoided or are enforced differently. This requires troubleshooting in a per-case basis, if they ever come up, but that comes with the territory.
All *State config data is now nested within the configs parameter, and aren't called states as they're to be considered static, per Server instance. The same data is also grouped as follows: configs > ( extended[] | facets > aggregations[] | table > columns[] | matchbox[] ) This change was made bearing in mind upcoming customisation capabilities for the facets panel, adding configs for other components that currently don't allow any, etc.
~Columns State's~ Table's defaultSorted is now defaultSorting.
The Table component doesn't include all its bells and whistles out of the box anymore, and users must add the extensions separately (e.g. toolbar, pagination, etc.)
the propertyfield is now renamed to fieldName to help differentiate when you're dealing with the whole object vs only its name. (e.g. no more field.field)

WIP Server Migration path: (figuring out how to phrase the steps, so for now it's just a partially incoherent mental datadump... bear with me)

Export your existing configs from a project, using admin-ui. These files will come out named and we'll need to make a couple of changes before they can be used. (mostly removing clutter, and renaming a field or two)
Consolidate these along with a new base.json (name is not mandatory, but advised as documentation will refer to it) file in a same folder. This new file will contain the document type (formerly known as GraphQL field) to be used by Arranger Components when querying the Server, and the ES index name for the Server's own needs. Furthermore, the Server codebase currently includes a configs folder that can be reused.
The files exported from Admin-UI will contain JSON objects that need be modified slightly. the following phrasing will really NEED improvement: Each object needs to be nested within another one, with a key derived from the filename. e.g. extended.JSON is {[...]}, becomes { "extended" : {[...]} }. The key will be used by the server to compose a large configs object, used pretty much everywhere internally. More detail needs to be added to explain this better. likely an example of what the overall config object looks like could be useful.
If using a different folder from the one included in the codebase, give your path to the Server container as you run its Docker image using CONFIG_PATH.
profit.

WIP Components Migration path: (Assuming the Server has been migrated)

If you're giving Arranger custom data fetching functions/middleware, remove any references to the admin and projects endpoints.
```
i.e. both `blah.com/admin/graphql` and `blah.com/<projectName>/graphql` are now just `blah.com/graphql`
```
Optional new feature You may add a query to the new hasValidConfig param. This helps verify the server has the right configs for a desired ES index and GraphQL field, and handle errors accordingly.
(WIP) replace <Arranger /> with Arranger's context <DataProvider/>, which is wired to add all the new providers necessary. Internally, the Arranger components that rely on this and other contexts already include their own hooks, etc; however, we expose useDataContext to be added in any custom components you want to access Arranger's data from. Thought we tried our best to make a straight swap-out viable, some setups may require a bit more intervention. i.e. The Data Provider only takes children, removing the component or render props. For those cases, your components would only require adding a consumer hook (see above); without the need for prop drilling shenanigans.
(Renamed) CurrentSQON is now SQONViewer (changed for declarativeness, and to avoid confusing the component with the value of the same name), and while the original name is still available for integration, it will be deprecated in a later version. A console warning will be displayed when the old name is used.
(Rewritten) There's a new Table, built upon the react-table v8 implementation. Its sibling components have also been reimplemented, and are now modular (e.g. you can compose your own toolbar using the individual elements, and have them both above and beneath the table). We're also reducing the myriad of flag configuration props, so it's easier to reason with this set of components. Customisation is now done through the theming system.
TO DO: Heaps of detailed docs need to be written still in regards to the theming system.

documentation of changes for Theming system in comments at #734

overture-stack / arranger