ygojson / ygojson-tools

Tooling for the ygojson project
MIT License
1 stars 0 forks source link

Change the driver of the tooling around the in-memory DB representation? #139

Open magicDGS opened 3 months ago

magicDGS commented 3 months ago

If we implement #138 as the way to keep in memory our models before we output to the final representation (the file representation of YGOJSON), the back and forth conversion between the fetched data and the final model would pass always through the DB model. This means that there would be unnecessary transfer of data between models without any value.

A better option is to drive the application by using our DB-entities instead as the driver of the application:

  1. Fetched data is converted into our DB-entities
  2. This DB-entities could be save directly
  3. The DB-entities can be requested and updated by using the other data sources (future development)
  4. No intermediate layer of our models would be required.

This means that we will keep the output layer completely detached from the fetch layer, with the DB in the middle to be the translation point. This makes more sense because most likely the cron-job will operate over entities that might have extra information that we don't want in the final models (i.e., unique-ID that is independent of the maybe missing UUID v5 that is consistent between runs from scratch).

Nevertheless, this also means that if we want to have the DB as the driver, this DB-entities might not be the right ones to create the DB-format outputs (but that will also be the case anyway). First we need to decide if we want that and then see if it makes sense with the current design (still private).

magicDGS commented 1 month ago

Will be done within the following iteration of changes:

magicDGS commented 1 month ago

Set mapping implemented with https://github.com/ygojson/ygojson-tools/pull/159