Closed dzxsll closed 6 years ago
dirk-thomas
commented
6 years ago Please post any kind of output as text - images are not searchable.
Please also provide exact steps to reproduce the problem. In this case the content of your workspace as well as the locate you have set in your environment.
dzxsll
commented
6 years ago Thanks for your answer. First I created a ROS2 package by using ros2 pkg create then I try to build it but failed. I just Installed Binary packages in C:\dev\ros2 and didn't change anything about the loacte. All location is same as the https://github.com/ros2/ros2/wiki/Windows-Install-Binary. And my package location is C:\dev\ros2\mySource Here is my error output. I'm sorry for that don't know how to correctly ask question. -------------output------------------------- c:\dev\ros2\mySource>colcon build Starting >>> helloworld [1.483s] colcon.colcon_core.shell ERROR Exception in shell extension 'bat': 'utf-8' codec can't decode byte 0xb8 in position 30: invalid start byte Traceback (most recent call last): File "c:\python37\lib\site-packages\colcon_core\shell__init__.py", line 231, in get_command_environment task_name, Path(build_base), dependencies) File "c:\python37\lib\site-packages\colcon_core\shell\bat.py", line 103, in generate_command_environment env = await get_environment_variables(cmd, cwd=str(build_base)) File "c:\python37\lib\site-packages\colcon_core\shell__init__.py", line 273, in get_environment_variables parts = line.decode().split('=', 1) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb8 in position 30: invalid start byte
[1.484s] colcon.colcon_cmake.task.cmake.build ERROR Could not find a shell extension for the command environment Failed <<< helloworld [ Exited with code 1 ]
Summary: 0 packages finished [0.56s] 1 package failed: helloworld
dirk-thomas
commented
6 years ago The information isn't sufficient to reproduce the problem yet. Please include how exactly you create the package (with all the arguments you used as well as potential other edits of the manifest file). Also please post the content of your environment variables (the output of set).
You could also edit the source file where the exception is happening and try to add some debug output in that case to see what the variable line contains in that case.
dzxsll
commented
6 years ago Only [ c:\dev\ros2\mySource ros2 pkg create helloworld ]
The output of set:
And thank you for your patience.
I will try to debug to see if I can find out the reason. PS: I tried to build a same package in my workmate computer and it worked.now I am so upset.
dzxsll
commented
6 years ago I find the problem in the Python37\Lib\site-packages\colcon_core\shell__init__.py and in the function get_environment_variables .
async def get_environment_variables(cmd, *, cwd=None, shell=True):
"""
Get the environment variables from the output of the command.
:param args: the sequence of program arguments
:param cwd: the working directory for the subprocess
:param shell: whether to use the shell as the program to execute
:rtype: dict
"""
output = await check_output(cmd, cwd=cwd, shell=shell)
env = {}
for line in output.splitlines():
print(line.rstrip())
line = line.rstrip()
if not line:
continue
parts = line.decode().split('=', 1)
if len(parts) != 2:
continue
env[parts[0]] = parts[1]
assert len(env) > 0, "The environment shouldn't be empty"
return envI print out the variable "line" and result is that " b'adb=D:\workInNingbo\appSource\\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools' "
So what does that mean? what I can do ?
calvertdw
commented
6 years ago My guess here would be that UTF-8 does not contain the Chinese character set. I'm not sure where the UTF-8 requirement is held.
calvertdw
commented
6 years ago The python decode method will require the correct encoding to be passed in, as it defaults to utf-8.
File "c:\python37\lib\site-packages\colcon_core\shell_init_.py", line 273, in get_environment_variables
parts = line.decode().split('=', 1)Can you please share information about your locale settings (by posting the output of the following commands):
python -c "import locale; print(locale.getlocale())"
python -c "import locale; print(locale.getdefaultlocale())"
python -c "import locale; print(locale.getpreferredencoding())"Thank you for both answer. My win10 is the Chinese version but all location are named without any Chinese character. Should I change the os character set?
the output : d:\dev\packages\1.helloworld>python -c "import locale; print(locale.getlocale())" (None, None)
d:\dev\packages\1.helloworld>python -c "import locale; print(locale.getdefaultlocale())" ('zh_CN', 'cp936')
d:\dev\packages\1.helloworld>python -c "import locale; print(locale.getpreferredencoding())" cp936
Let me try to change the character set and see what will happen.
dzxsll
commented
6 years ago oh, I get it. I see where have the Chinese characters. when the colcon build a package, every environment variable will be sent to check. However anyone chinese character can cause the question. Now I deleted the variable and it can work!!! I so appreciate your answer and patience.
dirk-thomas
commented
6 years ago It would be easy to patch the code to ignore environment variable which do contain characters which can't be decoded. Though I noticed that e.g. PATH contained chinese characters in your case and just skipping PATH wouldn't work. So it would be good to figure out how to make it work...
Can you please try the following command and post the output - it is trying to decode one of the problematic lines using your default encoding:
python -c "line=b'adb=D:\workInNingbo\appSource\\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools';line = line.decode('cp963'); print(line)"The command doesn't work , I am sorry. the cmd said LookupError:unkonwn encoding: cp963. But I can show you the string which contains chinese characters. Here: adb=D:\workInNingbo\appSource\附件四：灵云麦克风阵列调试软件\adb环境搭建工具包\platform-tools
it just mean attachment four: lingyun Microphone Array debug tool \ adb environment setup package
dirk-thomas
commented
6 years ago The locale had a flipped character. Please try this one again:
python -c "line=b'adb=D:\workInNingbo\appSource\\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools';line = line.decode('cp936'); print(line)"The result:
D:\dev\packages\helloworld>python -c "line=b'adb=D:\workInNingbo\appSource\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools';line = line.decode('cp936'); print(line)"
Traceback (most recent call last):
File "
Sometime the chinese characters are really big trouble.
dirk-thomas
commented
6 years ago That is weird that is can't even decode the string with the default locale of your system. If you can figure out what locale is required to decode the string correctly on your system I would be happy to work on a patch.
dzxsll
commented
6 years ago HI,I made it. The reason why it can't decode the string is that There are three lost "\" in the string
appSource\xb8 should be appSource\\\xb8
xfe\adb should be xfe\\adb
And It can work. I don't konw how it transforms between UTF-8 and cp936. But fact is that something lost.
Here are command and result: D:\dev\packages\helloworld>python -c "line=b'adb=D:\workInNingbo\appSource\\\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools';line = line.decode('cp936'); print(line)"
adb=D:\workInNingboppSource\附件四：灵云麦克风阵列调试软件\adb环境搭建工具包\platform-tools
dirk-thomas
commented
6 years ago And if you run the same command with utf-8 (or no argument) instead of cp936? Does it raise a UnicodeDecodeError again?
dzxsll
commented
6 years ago Result one:
D:\dev\packages\helloworld>python -c "line=b'adb=D:\workInNingbo\appSource\\xb8\xbd\xbc\xfe\xcb\xc4\xa3\xba\xc1\xe9\xd4\xc6\xc2\xf3\xbf\xcb\xb7\xe7\xd5\xf3\xc1\xd0\xb5\xf7\xca\xd4\xc8\xed\xbc\xfe\adb\xbb\xb7\xbe\xb3\xb4\xee\xbd\xa8\xb9\xa4\xbe\xdf\xb0\xfc\platform-tools';line = line.decode('utf-8'); print(line)"
Traceback (most recent call last):
File "
Result tow: D:\dev\packages\helloworld>python -c "line=b'adb=D:\workInNingbo\appSource\附件四：灵云麦克风阵列调试软件\adb环境搭建工 具包\platform-tools';line = line.decode('utf-8'); print(line)" File "string", line 1 SyntaxError: bytes can only contain ASCII literal characters.
I think the Result tow is what you want.
dirk-thomas
commented
6 years ago I think the Result tow is what you want.
No, result 2 is not valid since you can't use non-ASCII characters in bytes.
Anyway I think using locale.getpreferredencoding() for decoding the lines (rather than using the default utf-8) seems to solve the problem for you. I created #95 which should fix the problem.
It would be great if you could try the proposed patch locally and confirm if it works afterwards (using the environment for which it fails currently). To install the modified version of colcon-core you can run:
pip install -U git+https://github.com/colcon/colcon-core.git@use_encoding_for_env
I try to build a ROS2 package but failed. My os is win10 with python3.7.0. Here is my problem screent shot.