search

cloudquery / plugin-sdk

CloudQuery Go SDK for source and destination plugins
Mozilla Public License 2.0
22 stars 24 forks source link

Destination plugins receive escaped data #622

Closed candiduslynx closed 1 year ago

candiduslynx commented 1 year ago

Describe the bug

JSON fields (and possibly others) receive escaped data, as was seen in https://github.com/cloudquery/cloudquery/issues/7001. The reason for this is that source plugins use json.Marshal that does escaping, but the destination plugins can't perform unmarshaling due to lack of original structure. So, the data received has unicode characters escaped, and it might be written to the destination in the same way.

Expected Behavior

All values received by destination plugins should be unescaped.

Steps to Reproduce

  1. Create a tagged resource in Azure with key MyKey and value A&B.
  2. Run the destination plugin in debugger
  3. See that the value is received as {"MyKey":"A\u0026B"}, while it should've been presented as {"MyKey":"A&B"}

Possible Solution

Add preliminary unescape for the data in the destination.

Provider and CloudQuery version

latest

Additional Context

No response

candiduslynx commented 1 year ago

I suggest adding the Unicode characters to the generated input in the plugin testing for every applicable type. This way we'll be able to see the data types that require unescaping.

candiduslynx commented 1 year ago

Related to https://github.com/cloudquery/cloudquery/pull/7815