Version 6 is not handling special characters correctly

qetza / replacetokens-task

Azure Pipelines task to replace tokens in files with variables.

MIT License

11 stars 2 forks source link

Version 6 is not handling special characters correctly #38

Closed nubgamerz closed 2 months ago

nubgamerz commented 3 months ago

We are in the process of updating v3 to v6.

In a test of one of our pipelines, we have some data that has special characters like the Euro symbol €

The pipeline is failing, as this symbol is not being parsed correctly, and is instead being replaced with â‚¬

ReplaceTokens@3 handles this fine, it is only happening when using ReplaceTokens@6. The pipeline then thinks this is an escape character and breaks the process.

qetza commented 3 months ago

Hi @nubgamerz, There is a breaking change between v3 and v6 that could be the cause: you must now explicitly set addBOM to true if you want to add the BOM on unicode files. Maybe your file needs to have a BOM which is now not set by default on writing.

Could you share your configuring in v3 and v6 and the encoding of your original file?

nubgamerz commented 3 months ago

Thanks for getting back to me. I'm running a test with addBOM to true and so far it seems to be working.

Will need to run a few more tests with diagnostics enabled to be sure, but so far looks good.