Today we separately maintain a repository of source product data samples in XML format with notations including product, version and notes about how the sample was collected (lab, documentation, internet, etc.). There is no linkage between this critical information and the eventgen templates and config we maintain for the same source products in another repo. This is problematic given that when eventgen is used for QA of an add-on there is no ability to determine what source product or version/versions are being simulated. This means that if a bug/issue is found, manual investigation must happen to track down the versions in question, if that is even possible with the available information. Additionally, since there is no real visibility into the data used to create the templates the quality/trustworthiness of the resulting synthetic data is effectively unknown.
For example, events provided without information about the source product configuration and/or environment would be higher risk than events from a lab with well documented configuration.
The requested enhancement will provide a mechanism to centrally manage and store notated source data samples, eventgen templates and config for a source product. Additionally, the solution will have tracking for source products and versions.
As a Developer/Researcher/QA I may generate data needed to test a technical add-on against specific products/components and versions so that I may ensure the compatibility/support that is intended.
For example, I can generate events for Cisco ASA firewall events for version 9.13.
As a Developer/Researcher/QA I can easily see how the source events for specific products and versions were captured and which eventgen templates are based on these samples so that I may judge how trustworthy the resulting synthetic data is.
Today we separately maintain a repository of source product data samples in XML format with notations including product, version and notes about how the sample was collected (lab, documentation, internet, etc.). There is no linkage between this critical information and the eventgen templates and config we maintain for the same source products in another repo. This is problematic given that when eventgen is used for QA of an add-on there is no ability to determine what source product or version/versions are being simulated. This means that if a bug/issue is found, manual investigation must happen to track down the versions in question, if that is even possible with the available information. Additionally, since there is no real visibility into the data used to create the templates the quality/trustworthiness of the resulting synthetic data is effectively unknown. For example, events provided without information about the source product configuration and/or environment would be higher risk than events from a lab with well documented configuration.
The requested enhancement will provide a mechanism to centrally manage and store notated source data samples, eventgen templates and config for a source product. Additionally, the solution will have tracking for source products and versions.
As a Developer/Researcher/QA I may generate data needed to test a technical add-on against specific products/components and versions so that I may ensure the compatibility/support that is intended. For example, I can generate events for Cisco ASA firewall events for version 9.13.
As a Developer/Researcher/QA I can easily see how the source events for specific products and versions were captured and which eventgen templates are based on these samples so that I may judge how trustworthy the resulting synthetic data is.