Missing content encoding for attachments in JSON reports

[badeball]

👓 What did you see?

Consider the following code adding two string attachments:

Given("a step", function () {
  const myString = "foobar"
  this.attach(myString, "text/plain");
  this.attach(Buffer.from(myString).toString("base64"), "base64:text/plain");
});

This will create two envelopes (messages) with different contentEncoding (IDENTITY and BASE64 respectively). However, this information (content encoding) is lost in the JSON report and the original data cannot be reproduced with certainty.

Below is the JSON report:

[
  {
    "description": "",
    "elements": [
      {
        "description": "",
        "id": "a-feature;an-example",
        "keyword": "Example",
        "line": 2,
        "name": "an example",
        "steps": [
          {
            "arguments": [],
            "keyword": "Given ",
            "line": 3,
            "name": "a step",
            "match": {
              "location": "features/definitions.js:3"
            },
            "result": {
              "status": "passed",
              "duration": 492577
            },
            "embeddings": [
              {
                "data": "foobar",
                "mime_type": "text/plain"
              },
              {
                "data": "Zm9vYmFy",
                "mime_type": "text/plain"
              }
            ]
          }
        ],
        "tags": [],
        "type": "scenario"
      }
    ],
    "id": "a-feature",
    "line": 1,
    "keyword": "Feature",
    "name": "a feature",
    "tags": [],
    "uri": "features/foo.feature"
  }
]

✅ What did you expect to see?

I guess for JSON reports to either only ever contain base64 encoded data or for content encoding to be propagated to the report.

📦 Which tool/library version are you using?

• cucumber-js: 9.0.1

🔬 How could we reproduce it?

Steps to reproduce the behavior:

1. git clone https://github.com/badeball/cucumber-reproducible-issues.git
2. cd cucumber-reproducible-issues/string-attachment-ambiguity
3. npm install
4. npx cucumber-js

[badeball]

For reference, cucumber-json-formatter will take attachments with IDENTITY encoding and base64-encode them before outputting a JSON report, thus generating a different one, where both embeddings are identical.

[davidjgoss]

For reference, cucumber-json-formatter will take attachments with IDENTITY encoding and base64-encode them before outputting a JSON report, thus generating a different one, where both embeddings are identical.

Okay, that seems like the canonical behaviour we should follow then.

[badeball]

Excellent, I'll attempt to create a PR :+1:

[badeball]

This would however mean that string attachments, as they were attached pre #1552, would suddenly appear as base64-encoded data in JSON reports. Thus making this seem like a breaking change. Is that relevant in any way?

[davidjgoss]

Yep, I think it's a breaking change. We have the end of life for Node.js 14 coming up at the end of April so we could do a major release then with both in.

[davidjgoss]

Released in https://github.com/cucumber/cucumber-js/releases/tag/v10.0.0

[v-mwalk]

Note that this was actually a breaking change. We attach Playwright screenshots to the JSON for our HTML reports, base64 encoding before attaching. Very confused when there appeared to be a double Base64 encode going on! lol

[badeball]

@v-mwalk, you’re right that it’s a breaking change. The previous conversation also supports this. I think the only mistake made is that it wasn’t mentioned as such in the release notes.