Open Appointat opened 1 year ago
Hm.. I develop on Windows and do not encounter this error. I normally run stuff in Anaconda powershell console.
Thanks. I tried pre-commit
on Linux, and it works too. It could be the error of my Windows env or default settings. When I resolve the issue, I will put the feedback here.
Hi, I found this solution works for me on Windows 11:
To change the default character encoding in Windows, you need to modify Python's locale settings. Python uses the locale library to handle locale-related tasks such as character encoding, number, and date formats, etc. This is a somewhat advanced operation and may affect all Python programs on your system.
In Python 3.7 and later versions, you can globally set Python to use UTF-8 encoding by default in Windows environments by setting the PYTHONUTF8
environment variable to 1
.
Here are the steps to do that:
Win+X
, and select System
.About
, then on the right, select System info
.Advanced system settings
.Environment Variables
.New
below, and in the new row, input PYTHONUTF8
and 1
.Then click OK
, close all dialog boxes.
Restart your command prompt or PowerShell window, Python will use UTF-8 as the default character encoding.
Please note that this method will change the default encoding method for all Python programs. If some programs depend on GBK or other encodings, unpredictable problems may occur. You need to ensure you understand the impact of this operation and know how to restore settings if something goes wrong.
It seems unrelated to the project itself but more like a common pitfall for contributors with a Chinese-English working environment. Maybe introducing a docker container or VSCode Dev container in the workflow could eliminate such issues from the root.
It seems unrelated to the project itself but more like a common pitfall for contributors with a Chinese-English working environment. Maybe introducing a docker container or VSCode Dev container in the workflow could eliminate such issues from the root.
Introducing a docker container sounds great. Thanks @kuang-da for the suggestion!
Hi, I found this solution works for me on Windows 11:
To change the default character encoding in Windows, you need to modify Python's locale settings. Python uses the locale library to handle locale-related tasks such as character encoding, number, and date formats, etc. This is a somewhat advanced operation and may affect all Python programs on your system.
In Python 3.7 and later versions, you can globally set Python to use UTF-8 encoding by default in Windows environments by setting the
PYTHONUTF8
environment variable to1
.Here are the steps to do that:
- Press
Win+X
, and selectSystem
.- Click on
About
, then on the right, selectSystem info
.- In the list on the left, choose
Advanced system settings
.- In the System Properties dialog, select
Environment Variables
.- In the Environment Variables dialog, click on
New
below, and in the new row, inputPYTHONUTF8
and1
.Then click
OK
, close all dialog boxes.Restart your command prompt or PowerShell window, Python will use UTF-8 as the default character encoding.
Please note that this method will change the default encoding method for all Python programs. If some programs depend on GBK or other encodings, unpredictable problems may occur. You need to ensure you understand the impact of this operation and know how to restore settings if something goes wrong.
Thank you very much for your method! I am also a windows11 user and a contributor with Chinese-English working environment. And I met the same error and found that your solution is useful. By the way, to solve this problem temporarily, inputing 'set PYTHONUTF8=1
' befor input 'git commit ...
' in command prompt or PowerShell window is also a convenient solution.
@yiyiyi0817 Glad to hear that it is helpful for you.
Required prerequisites
What version of camel are you using?
0.1.0
System information
>>> import sys, camel >>> print(sys.version, sys.platform) 3.8.10 (tags/v3.8.10:3d8993a, May 3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] win32 >>> print(camel.version) 0.1.0
Problem description
Here is a sample issue written according to your requirements:
Title: Encoding issues with UTF-8 and GBK across different systems
Problem Description:
I am currently using camel version x.y.z on a Windows system. When attempting to run pre-commit checks using the update_license.py script, I encountered an error. This error appears to be due to an encoding mismatch - while my system is defaulting to GBK, the script seems to be expecting files encoded in UTF-8.
This issue specifically occurs when the script attempts to open and read from a file. The error message received is:
The expected behavior is for the script to successfully read from the file and execute the pre-commit checks. However, due to the encoding mismatch, this is not happening.
Reproducible example code
Reproducible Example Code:
Python Snippets:
Unfortunately, without knowing the exact content of your
update_license.py
script, I can only provide a generic example of where the issue may arise. The issue most likely occurs when the script attempts to read a file:If
file.txt
is encoded in UTF-8 but the system defaults to GBK, this will raise aUnicodeDecodeError
.Command Lines:
This issue is encountered when running pre-commit checks using the
update_license.py
script:Extra Dependencies:
No additional dependencies are necessary to reproduce this issue. However, ensure you are using the correct version of Python and that you have all necessary packages installed.
Steps to Reproduce:
update_license.py
script.UnicodeDecodeError
.Traceback