As a developer
I want to monitor how long it takes to process a notebook with different cluster settings
So I continously have an idea on how long it would take to process a predefined dataset
AC 1: a predefined timeseries dataset is available in the landing-zone (bulk load)
AC 2: record how many seconds it took to ingest timeseries into bronze-tables for timeseries
AC 3: record how many seconds it took to transform timeseries into silver-tables for timeseries
AC 4: record how many seconds it took to map timeseries into gold-tables for timeseries
AC 5: record metadata about the cluster used for the testrun
AC 6: at least the 2 latest testruns using different clusters configuration should be available for monitoring
AC 7: should be able to report monitoring result in a non-jiberish way (ex. a graphical dashboard or in plain text)
Story:
As a developer I want to monitor how long it takes to process a notebook with different cluster settings So I continously have an idea on how long it would take to process a predefined dataset
AC 1: a predefined timeseries dataset is available in the landing-zone (bulk load) AC 2: record how many seconds it took to ingest timeseries into bronze-tables for timeseries AC 3: record how many seconds it took to transform timeseries into silver-tables for timeseries AC 4: record how many seconds it took to map timeseries into gold-tables for timeseries AC 5: record metadata about the cluster used for the testrun AC 6: at least the 2 latest testruns using different clusters configuration should be available for monitoring AC 7: should be able to report monitoring result in a non-jiberish way (ex. a graphical dashboard or in plain text)