search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Get access to the Salt software package repository here:
https://repo.saltproject.io/
Apache License 2.0
14.11k stars 5.47k forks source link

[BUG] `salt-call --local -g --out yaml` redirected stdoutput prints bytestring on Simplified Chinese Windows #62399

Open zraul123 opened 2 years ago

zraul123 commented 2 years ago

Description Given a Windows with the locale on Simplified Chinese, or Taiwanese (they have different cp so fixing one might not fix the other) If you run salt-call --local grains.items --out yml (reproduces without --out yml or with --out json) and redirect the output (programatically or using the > file.txt in command-line/powershell) the output is a binary-string as a string.

Example: image

The problem we're facing is that we're using salt-call grains.items --out yml from a C# Application, redirect the Process stdout, and then we try to deserialize the values using an YamlDotNet. The deserialization fails because we're getting the b"local:..." (with b" as the first characters and \n as escaped characters) as a string so it's not a valid YAML.

Important findings If you go to salt/utils/yamldumper.py and change the line 127 to kwargs["allow_unicode"] = False

Setup

This has been reproduced both on an actual machine and a VM.

Steps to Reproduce the behavior Easiest way to reproduce it:

This has been reproduced on many salt versions, some have different outputs than others:

Expected behavior

Screenshots image

Versions Report

salt --versions-report (Provided by running salt --versions-report. Please also mention any differences in master/minion versions.) ```yaml Salt Version: Salt: 3004 Dependency Versions: cffi: 1.14.6 cherrypy: 18.6.1 dateutil: 2.8.1 docker-py: Not Installed gitdb: 4.0.7 gitpython: 3.1.18 Jinja2: 2.10.1 libgit2: Not Installed M2Crypto: Not Installed Mako: 1.1.4 msgpack: 0.6.2 msgpack-pure: Not Installed mysql-python: Not Installed pycparser: 2.20 pycrypto: Not Installed pycryptodome: 3.10.1 pygit2: Not Installed Python: 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:18:16) [MSC v.1928 64 bit (AMD64)] python-gnupg: 0.4.7 PyYAML: 5.4.1 PyZMQ: 19.0.0 smmap: 4.0.0 timelib: 0.2.4 Tornado: 4.5.3 ZMQ: 4.3.2 System Versions: dist: locale: cp1252 machine: AMD64 release: 10 system: Windows version: 10 10.0.22000 SP0 Multiprocessor Free ```

Additional context These are all my findings..

welcome[bot] commented 2 years ago

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

zraul123 commented 2 years ago

@dwoz any updates on this?

zraul123 commented 2 years ago

@dwoz @garethgreenaway If you go to salt/utils/yamldumper.py and change the line 127 to kwargs["allow_unicode"] = False (which defaults to, due to calling with salt-call, is there a way to set this another way?) resolves this problem, but I'm not fully comprehending what the impact on the whole codebase is (since it's so big)

Can any of you confirm that this is the correct solution and proceed with a PR, or otherwise point me in the right direction? Thank you!