[Meta] corpus generator tool enhancements

shmsr commented 6 months ago

Introduction

The elastic-integration-corpus-generator-tool is a valuable resource for generating test corpus for testing & benchmarking. To improve its usability and functionality, we have identified several areas for enhancement in this issue.

Notes

Enum Support:
- Currently, enum is only supported for the keyword type.
- Extend enum support to other relevant types, such as the long type.
- Use case: http-response-status-code of long type could benefit from enum support.
- Implement support for weighted enum as proposed in elastic/elastic-integration-corpus-generator-tool#130.
Configuration Simplification:
- Generating Schema B currently requires multiple config files.
- Simplify the configuration by adding a types section to config.yml.
- Allow either the types section or fields.yml, but not both. Raise a validation error if both are present to avoid ambiguity.
- Update the package-spec to make fields.yml optional.
- The goal is to provide a minimal configuration to start the corpus generator.
Counter Type Enhancements:
- Counter support was recently added to the tool.
- Introduce a counter-reset feature for the counter type.
- See elastic/elastic-integration-corpus-generator-tool#124 for more details.
Formatting Pattern Feature:
- There have been requests to add helper methods like path and hostnames, but adding individual helpers is not sustainable.
- Introduce a formatting_pattern feature to provide a more flexible and extensible solution.
- See the discussion in elastic/integrations#8781 (comment) for more context.
Clarification Needed:
- Seek clarification on the discussion in https://github.com/elastic/integrations/pull/8781#discussion_r1433886891
- Seek clarification on the discussion in https://github.com/elastic/integrations/pull/8781#discussion_r1433869402
- I understand the point to remove these variables and instead they should be generated directly just by saying $agentName for example. For example if there is again a field that user defines named agentName with type long then that would be a collision as we have reserved the agentName as ECS field. Partly saying, we need to mention that for ECS fields there is a reserved naming convention for example host.id -> hostId which should not be generated again for some other field generation. Hope this is only the point discussed here.
Return better error messages
- https://github.com/elastic/elastic-integration-corpus-generator-tool/issues/123

Related PRs and Issues:

Notes are extracted from GitHub issues and the related PR linked. From the notes itself, some tasks have been created to get started with the enhancements for the tool. Also, there few more issues in the PR linked from which we have to extract notes and tasks.

Tasks

[x] https://github.com/elastic/elastic-integration-corpus-generator-tool/issues/142
[ ] https://github.com/elastic/elastic-integration-corpus-generator-tool/issues/130
[ ] https://github.com/elastic/elastic-integration-corpus-generator-tool/pull/149
[x] Add counter-reset feature to the counter type
[ ] Add formatting pattern support
[ ] https://github.com/elastic/elastic-integration-corpus-generator-tool/issues/123
[ ] General enhancements and bug-fixes

CC: @tommyers-elastic @ruflin

ali786XI commented 6 months ago

Couple of more issues:-

shmsr commented 6 months ago

Couple of more issues:-

Use elastic-package benchmark stream outside package directory elastic-package#1640

List packages with benchmarks elastic-package#1634

Thanks! 🙌🏽

elastic / elastic-integration-corpus-generator-tool