[BUG]UnicodeDecodeError: 'gbk' codec can't decode byte 0xac in position 102: illegal multibyte sequence

[Jontyshaw]

Description

When loading a variable from the yaml environment, the crewai library tries to load it in gbk encoding as soon as Chinese is present in it, but my yaml file is utf-8. You want to add a measure to detect file encoding when loading yaml files

Steps to Reproduce

1. edit agent.yaml or task.yaml
2. Write a lot of Chinese into the above two yaml files
3. crewai run

Expected behavior

When loading yaml files, it automatically detects whether the file encoding is gbk or utf-8 or something else, and loads the file in this encoding format because it involves the definition of agents and the definition of tasks, and people in many other countries may use different encodings

Screenshots/Code snippets

Operating System

Windows 11

Python Version

3.12

crewAI Version

0.51.0

crewAI Tools Version

0.51.0

Virtual Environment

Poetry

Evidence

` crewai run C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\pydantic_internal_config.py:341: UserWarning: Valid config keys have changed in V2:

• 'allow_population_by_field_name' has been renamed to 'populate_by_name'
• 'smart_union' has been removed warnings.warn(message, UserWarning) Running the crew C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\pydantic_internal_config.py:341: UserWarning: Valid config keys have changed in V2:
• 'allow_population_by_field_name' has been renamed to 'populate_by_name'
• 'smart_union' has been removed warnings.warn(message, UserWarning) Traceback (most recent call last): File "", line 1, in File "C:\Users\jonty\PycharmProjects\crew_agent\src\crew_agent\main.py", line 22, in run CrewAgentCrew().crew().kickoff(inputs=inputs) ^^^^^^^^^^^^^^^ File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\crewai\project\crew_base.py", line 38, in init self.agents_config = self.load_yaml( ^^^^^^^^^^^^^^^ File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\crewai\project\crew_base.py", line 53, in load_yaml return yaml.safe_load(file) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml__init__.py", line 125, in safe_load return load(stream, SafeLoader) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml__init.py", line 79, in load loader = Loader(stream) ^^^^^^^^^^^^^^ File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml\loader.py", line 34, in init Reader.init(self, stream) File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml\reader.py", line 85, in init__ self.determine_encoding() File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml\reader.py", line 124, in determine_encoding self.update_raw() File "C:\Users\jonty\AppData\Local\pypoetry\Cache\virtualenvs\crew-agent-OgCTPeQm-py3.12\Lib\site-packages\yaml\reader.py", line 178, in update_raw data = self.stream.read(size) ^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'gbk' codec can't decode byte 0xac in position 102: illegal multibyte sequence An error occurred while running the crew: Command '['poetry', 'run', 'run_crew']' returned non-zero exit status 1.

`

Possible Solution

When loading yaml files, it automatically detects whether the file encoding is gbk or utf-8 or something else, and loads the file in this encoding format because it involves the definition of agents and the definition of tasks, and people in many other countries may use different encodings

Additional context

no

[kustcl]

hello,have your questions has finished?,I also meet the same question,can you help me

[guojingfeng1994]

When I use custom i18n json files, the json files also have this problem when loading

[Jontyshaw]

No, it hasn't been resolved yet. Currently, it is executed by converting the yaml file to GBK format, or using it entirely in English.