I suggest expanding on the AQ.FLOW section with a data processing flow diagram that show how each of the data processing components follow. Specifically, describe visually and in words what the Pipeline Runner does in each of its stages. This also should help a developer understand what is going on with the data lineage. You may want to connect this to the database schema sections as well.
I suggest expanding on the AQ.FLOW section with a data processing flow diagram that show how each of the data processing components follow. Specifically, describe visually and in words what the Pipeline Runner does in each of its stages. This also should help a developer understand what is going on with the data lineage. You may want to connect this to the database schema sections as well.