search

cucumber / messages

A message protocol for representing results and other information from Cucumber
MIT License
18 stars 16 forks source link

Consider replacing the CCK with a testdata folder #110

Open mpkorstanje opened 1 year ago

mpkorstanje commented 1 year ago

A bit of a chicken and egg problem with the perl tests here - they rely on the current published version of the CCK, whose messages don't include the new field, which is causing the failure. Any thoughts on how to address this @ehuelsmann?

@davidjgoss looking at the test I think we can change them a little so. So instead of comparing the round trip ndjson --> object --> ndjson and comparing the result we can do object --> ndjson --> object and compare that result. That way we don't need the CCK at all and the code will change along with that specific message object.

Though I think it would be even better to include a testdata folder with (for now) a single message that all implementations try to round trip. That way we won't have to update N implementations manually.

Originally posted by @mpkorstanje in https://github.com/cucumber/messages/issues/102#issuecomment-1318916040

ciaranmcnulty commented 1 year ago

I think this is a good idea; how would the testdata be generated?

mpkorstanje commented 1 year ago

I think hand picking a few interesting cases from the current CCK should be sufficient.

For this repo we are mostly concerned with ensuring the serialization works. This means that both legs of a round trip work.

Wether or not the contents of a message is correct and consistent with the other messages in a report file isn't a concern here. So we don't need a cucumber implementation to generate them.

ciaranmcnulty commented 1 year ago

I agree, pick some from CCK + https://github.com/cucumber/messages/blob/main/java/src/test/java/io/cucumber/messages/NdjsonSerializationTest.java and put them in a folder and we can just check encode->decode->encode works and results in the same things