search

lukashornych / evitalab

Official web-based GUI client for evitaDB e-commerce database. It is built to help developers who use evitaDB with exploring and debugging their domain structure and data. Besides standard query executing, it also supports multiple no-code tools to quickly navigate through domain structure, docs and data without needing to write any queries.
https://github.com/lukashornych/evitalab
Apache License 2.0
7 stars 1 forks source link

Management tooling - Time machine #95

Open novoj opened 5 months ago

novoj commented 5 months ago

Subissue of #117

New version of evitaDB will be able to reason about different versions of the catalog, which are directly related to certain transactions (some of them could be grouped together - but still the granularity will be quite fine). evitaLab should introduce a new "menu item" called "Time Machine". It will allow to browse through all available versions of the catalog stored on the file system - for each record there will be information about

For each of these versions, the catalog will be able to provide a list of changes made in a particular transaction (transaction, if several are collapsed within it). But this will be a more expensive operation because the WAL would have to be accessed, read, and aggregated. Also, the information will be missing if the WAL has rotated in the meantime and the information is no longer accessible.

So the UI should provide means to focus on a certain time period and list a few (10-20) versions of the catalog from that time period. For each of this period, it should try to access the WAL and create a digest of the changes made in that version.

Time Machine should be able to restore the snapshot of the catalog in one of these particular catalog versions into a new catalog and load it into a store so that the data becomes available for querying.

lukashornych commented 5 months ago

So if I understand it correctly, the time machine screen will only list the changes and versions of catalogs but won't allow for reading actual data of particular version of a catalog in grid and schema viewers. But there will be functionally (a button?) which will be able to create a new catalog with custom name with restored data from a specific version of catalog. Then this catalog will be separate item in the main menu and will allow for data browsing?

novoj commented 5 months ago

Yep, that's right.

We wouldn't be able to easily access the data because it would require loading a significant amount of data into memory, and it wouldn't be feasible for multiple versions. But if the developer decides he wants to try to restore a particular version (and look at the data/schema, etc.), he restores it to a new catalog and either deletes that catalog and restores another one, or proceeds with the restored version. Still, I think it's a pretty impressive feature to have.

... and of course he could reason about the contents from the digest of the WAL.

lukashornych commented 2 weeks ago

Proposal of catalog history view (mutation listing with filter)

Image

Local mutation type will group mutations to: attribute, price, references, etc mutations, not specific mutations

novoj commented 2 weeks ago

The entities should be indented within the transaction to make the listing easier to follow. What about paging? Or maybe infinite load would be better here?!

lukashornych commented 2 weeks ago

That was first version, but than I looked into the codebase where the tx mutations separate. Will need to merge it in UI then.

Regarding paging, this will be tricky due to the grouping. From performance standpoint, I think traditional paging will be better but we need to figure out how to page by entire transactions.

lukashornych commented 2 weeks ago

Proposal of record history in entity viewer:

Frame 8

It's content will change based on targeted cell.

Entity history (not sure what to display here):

Image

Nested records history (attributes, prices, etc):

Image