yaml / yaml-test-suite

Comprehensive, language independent Test Suite for YAML
MIT License
172 stars 58 forks source link

2LFX and 6LVF appear identical #43

Open am11 opened 5 years ago

am11 commented 5 years ago

The only difference between 2LFX and 6LVF is a linebreak before "foo in in.yaml and out.yaml. Is it redundant or is this to assert that downstream emitter should respect linebreak before "foo (even though the expected event streams are identical)?

perlpunk commented 5 years ago

I should note that out.yaml in general is not yet correct in every case. We don't even have a definition on what it should contain. I started to use emit.yaml for emitting the parse results without loading. So don't take out.yaml too seriously. You can use emit.yaml for re-parsing and then compare the parsing events to the original parsing events. Those should be equal.

The two tests you are talking about: 2LFX was added by @flyx for YAML 1.3. There are several test cases which were copied and adapted to 1.3 (although we don't have a final decision on 1.3 yet). One rule for 1.3 says that top level scalars should always start on the next line after ---.

The linebreak is not part of the event, so it should not be respected.

am11 commented 5 years ago

Many thanks for the detailed reply!

One rule for 1.3 says that top level scalars should always start on the next line after ---.

I suppose when this rule will take effect, it will change the output (test.event) of 6LVF to:

 +STR
- +DOC ---
+ +DOC
- =VAL "foo
+ =VAL :--- "foo
 -DOC
 -STR

You can use emit.yaml for re-parsing

This is a very useful information! Incidentally, it seems like there are only 22 tests with emit.yaml, and 205 with out.yaml. (assuming it is not a manual process,) Would it be possible to generate emit.yaml for more (all?) tests? :)

and then compare the parsing events to the original parsing events.

Alright, I will try this. At present, we are reading in.yaml and asserting that parser generates an event stream matching the contents of test.event.

In addition to parsing (scanner+parser), I proposed to align YamlDotNet's emitter with what is expected by yaml-test-suite. Thinking from the perspective of emit.yaml, we can also parse in.yaml, save the YamlEventStream object as yaml content and assert the content be equal to emit.yaml. Would this (additional) comparison be realistic for downstream implementer?


The linebreak is not part of the event, so it should not be respected.

Thank you again for confirmation! Then YDN's scanning logic complies by skipping the linebreak in this case. It was for the curiosity; whether we need to remember this linebreak for output emitters.

perlpunk commented 5 years ago

The test 6LVF will be invalid in YAML 1.3 with this rule (because three dashes and a space are always a document header and never content).

The test cases were (and are) produced manually and automatically.

The problem is we don't have a perfect parser and emitter (yet) ;-) We usually use libyaml for generating the test.event and out.yaml, but when it fails, we have to write it manually.

I wrote all emit.yaml manually, by copying out.yaml and adapting it.

The idea is: When there is an emit.yaml, emitter output should look like it, but it is not the only correct output. If there is no emit.yaml, the emitter output should be the same as in.yaml.

But this part is work in progress, so still a lot of test cases probably need an emit.yaml.

Another thing which needs to be done is, mapping and sequence events need the information about flow or block style. As soon as this information becomes part of the events and test.event, also, emit.yaml will have to change.

So, there's still a lot todo!