search

openalm / Extension-UtilitiesPack

Release Management utility tasks
Other
34 stars 38 forks source link

UTF-8 encoded files gets ASCI encoding after tokenization #57

Closed henrikthorin closed 7 years ago

henrikthorin commented 7 years ago

Hi,

We use your Tokenizer to update configuration settings in VSTS. Our configuration files are UTF-8 and includes special characters. When run thru the tokenizer the characters gets messed up and the file encoding changes from UTF-8 to ASCI.

We've analysed your code and we believe that you're missing the encoding flag here Set-Content $tempFile -Force

Best regards Henrik

wkz123 commented 7 years ago

+1 just stumbled on this issue while transforming a config file :( Best regards

harshil93 commented 7 years ago

@honk82 Thanks for reporting it. I will fix it.

rbengtsson commented 7 years ago

We also have this issue. Any ETA on this one? :-) Thanks!

harshil93 commented 7 years ago

Anyone here has a fix for this ? I was going through the docs and found that the default encoding for the Get-Content cmdlet is ASCII.

I will have to first somehow detect the encoding of the file and then use the same encoding while saving it back.

harshil93 commented 7 years ago

Fixed in the latest commit for BOM encoded UTF files. Will update the extension on the marketplace. If you are using it in your on premise environment. Download the 0.1.5 version vsix from the root of the repo and use it.

thomazmoura commented 7 years ago

@harshil93 , do you know if this fix needs the v2 build agent like the secret variables one?

I'm currently using the 1.6 version of the extension with on-premises TFS 2015 U3 (build agent 1.95.3) and I'm having encoding problems whenever I run Tokenizer on Web.config's which have words with special characters like "Homoloção" (which are often used on XML comments).

I used Notepad++ to check the encoding formats before and after running Tokenizer and the original is UTF8 (with BOM) and the end result is ANSI. I'm using Tokenizer on a Release task directly on the deployed file, if that is of any help.