KaotoIO / kaoto

The UI of the Kaoto project
https://kaoto.io
Apache License 2.0
29 stars 23 forks source link

Write the catalog files at the end of the process #1250

Open lordrip opened 1 month ago

lordrip commented 1 month ago

Please describe the task that needs to be done

Currently, the catalog generation workflow looks like the following:

  1. Fetch the catalog information
  2. Write each piece to disk as soon as it completes
  3. The final result gets assembled into the index.jsonfile

The idea is to generate everything in memory and as the last step, write everything to file. This to be able to run assertions against the output more easily.

Todo

lordrip commented 4 weeks ago

Hi @shivamG640, I have a proposal for the catalog-generator test:

Extract each of these methods to individual classes, under a dedicated package The method could not receive any parameter, but instead, return an index entry that will be collected by the CatalogGeneratorBuilder class

By doing this, we can assert over the output of each class while also being able to mock them when needed to test the main class.

We should be able to postpone writing the files as a much as possible, up to the generate command.

lordrip commented 4 weeks ago

@mhempleman As you already noticed, we're adding tests for the catalog generator, and as part of this effort, we're thinking of reorganizing a few classes to make them more testable.

We're collecting proposals for this issue, and since you're contributing to the catalog generator, we wanted to give you a heads-up. This won't affect

I don't think it will affect https://github.com/KaotoIO/kaoto/pull/1311, since whatever is decided, it should be implemented on a step-by-step basis.

That being said, if you have a proposal or feedback about what changes would be nice to implement to make the catalog generator more robust and testable, please let us know so we can incorporate your feedback.