pantoniou / libfyaml

Fully feature complete YAML parser and emitter, supporting the latest YAML spec and passing the full YAML testsuite.
MIT License
239 stars 73 forks source link

Setup testing for the event creation in streaming mode #110

Closed kdubb closed 1 month ago

kdubb commented 1 month ago

During my investigation of #109, I determined that the emitter, in streaming mode, is missing testing of the event creation code path.

It's important to note that the current testing, while comprehensive in some aspects, does not cover the scenario where events are created via the fy_emit_event_create (or equivalent) API. This limitation underscores the necessity of the proposed solution.

To thoroughly test the event creation API code path, I added a new option, --recreating, to fy-tool when in dump mode. When enabled, each of the events emitted by the parser is recreated by calling fy_emit_event_create with the arguments coming from the parsed event.

Additionally, I added another set of tests, named restreaming, that adds both the --streaming and --recreation flags. This ensures that the existing code path, which tests emitting stream events that were parsed, and the stream event creation code path are tested.

With this new test set, there are 25 errors produced.

Notably, this comprehensive testing now brings to light the function fy_analyze_scalar_content which seems to be a significant contributor to the issues at hand. As an example, a scalar string, with only whitespace, is flagged with both FYACF_BLOCK_PLAIN & FYACF_FLOW_PLAIN. This results in all-whitespace strings being stripped of whitespace characters and produces an empty string.

kdubb commented 1 month ago

After fixing issues in the test harness, the number of failing tests was reduced to 15.

pantoniou commented 1 month ago

Interesting...

Yes testing of the raw event creation and emission is lacking. I will have to take a more thorough look, but at first glance looks good.

Good job.

pantoniou commented 1 month ago

Manually merged this (along with some minor fixes) with 39327405344e61e03f4c5cc318aab8bd30a6e7e7

Please test and verify