Activity failure is not encrypted with the key set to the context propagator, used to encrypt workflow and activity payload

[antmendoza]

Using this code as example, I have set the Failure converter with EncodeCommonAttributes to encrypt failures

c, err := client.Dial(client.Options{
...
...
FailureConverter: temporal.NewDefaultFailureConverter(temporal.DefaultFailureConverterOptions{
    EncodeCommonAttributes: true,
    DataConverter:          dataConverter,
}),

workflow and activity payload are encrypted with the key set to the context propagator but not failures.

Expected Behavior

Failure attributes should be encrypted

Actual Behavior

Failure attributes are not encrypted (or are encrypted with a different key)

Steps to Reproduce the Problem

There is a repro here and the workflow history that it produces where encryption-key-id is empty

[Quinn-With-Two-Ns]

As discussed offline this is not excepted to work because failure converters cannot be context aware. I opened up an issue to add support to failure converters https://github.com/temporalio/sdk-go/issues/1609