search

arduino / arduino-ide

Arduino IDE 2.x
https://www.arduino.cc/en/software
GNU Affero General Public License v3.0
2.23k stars 379 forks source link

Unicode/Locale compilation error #1513

Open exipilis opened 1 year ago

exipilis commented 1 year ago

Describe the problem

Compilation error:

Traceback (most recent call last):
  File "__main__.py", line 317, in <module>
  File "click/core.py", line 1134, in __call__
  File "click/core.py", line 1040, in main
  File "click/_unicodefun.py", line 100, in _verify_python_env
RuntimeError: Click will abort further execution because Python was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/unicode-support/ for mitigation steps.

This system lists some UTF-8 supporting locales that you can pick from. The following suitable locales were discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8
[38295] Failed to execute script __main__

exit status 1

Compilation error: exit status 1

Tried setting locale to uk_UA.UTF-8, en_US.UTF-8, C.UTF-8 etc. No effect.

To reproduce

  1. File -> Examples -> 01. Basics -> Blink
  2. Click Verify.

Expected behavior

Compilation success.

Arduino IDE version

Original report

2.0.0

Last verified with

2.0.3

Operating system

macOS

Operating system version

12.2, 13.1

Additional context

Additional reports

Issue checklist

per1234 commented 1 year ago

Hi @exipilis. Thanks for your report.

Which board do you have selected in the Tools > Board menu in the Arduino IDE when you get this error?

What is the normal locale on your system?

exipilis commented 1 year ago

@per1234 it was nRF52840 Feather Sense by Adafruit.

per1234 commented 1 year ago

It seems the problem is caused by a misconfiguration of the locale on your computer. There is some information about it here:

https://click.palletsprojects.com/en/8.1.x/unicode-support/#surrogate-handling

The problem is either with your system, the 3rd party "adafruit-nrfutil" tool used for compilation and uploading specific to your board, or perhaps even the upstream Nordic Semiconductor "nRF Util" project forked by Adafruit.

Since this does not seem to be directly related to the Arduino IDE codebase hosted in this repository, this issue tracker will not be an appropriate place for further discussion of the issue. You are welcome to request assistance over on the Arduino forum though:

https://forum.arduino.cc/

exipilis commented 1 year ago

@per1234 it works perfectly on Arduino IDE 1.8.19 somehow.

per1234 commented 1 year ago

Thanks @exipilis. I apologize for taking so long to circle back to this.

I previously had only tested it on the Windows and Linux machines that are accessible to me and was unable to reproduce the bug there. I finally tried it on macOS and sure enough I can reproduce the bug, as well as your finding that it doesn't occur with Arduino IDE 1.x.

I did some investigation. When I run the locale command from a terminal on the macOS machine, I get the expected output:

LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

Then I modified the boards platform to run the locale command as part of the compilation process by adding the following "pre-build hook" line to ~/Library/Arduino15/packages/adafruit/hardware/nrf52/1.3.0/platform.txt (you can actually add it to the platform.txt of any boards platform; there is nothing special about this specific platform in this regard):

recipe.hooks.prebuild.1.pattern=locale

After that modification, when I compile for a board of that platform, the locale command is executed as part of the compilation process and the following is printed in the Arduino IDE 2.0.0 verbose compilation output:

LANG=""
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

When I do the same on my Windows and Linux machines, the locale output is just the same whether it is executed from a terminal or from the compilation process.

When I do the same using an arduino-cli compile command from the command line on macOS, I get the identical output as I did from the terminal.

When I do the same using Arduino IDE 1.8.19, the output is similar to what I got from Arduino IDE 2.0.0:

LANG=""
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

The difference is that LC_CTYPE is set to "C" in the 2.0.0 output and set to "UTF-8" in the 1.8.19 output.

exipilis commented 1 year ago

@per1234 thanks for looking into. I tried adding 

recipe.hooks.prebuild.1.pattern=locale

to ~/Library/Arduino15/packages/adafruit/hardware/nrf52/1.3.0/platform.txt. Nothing changed, I still get the same error message during compilation on IDE 2.0.0.

Running locale before and after the compilation I see no difference:

LANG=""
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

Did I understand you correctly?

per1234 commented 1 year ago

Nothing changed, I still get the same error message during compilation on IDE 2.0.0.

That is expected. I added the line solely as an experiment to see what the locale settings were during the compilation in order to better understand the problem. It was not intended to fix the compilation error.

Running locale before and after the compilation I see no difference:

Is the output you shared from running locale from a terminal, or was it from the verbose compilation output?

ghost commented 1 year ago

@per1234 - I'm having the same issue. On OSX, Arduino 2.0.2 and 1.3 of the Adafruit nRF52-package.

Board: Adafruit Circuit Playground Bluefruit. (I have a Circuit Playground Express, and that works fine.)

Running locale on the command line I get: 

LANG=""
LC_COLLATE="en_GB.UTF-8"
LC_CTYPE="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_ALL="en_GB.UTF-8"

I've added the prebuild-flag to platform.txt and enabled verbose output, and during build I get


LANG=""
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

If I run the command on the command line

/Users/XXX/Library/Arduino15/packages/adafruit/hardware/nrf52/1.3.0/tools/adafruit-nrfutil/macos/adafruit-nrfutil dfu genpkg --dev-type 0x0052 --sd-req 0x00B6 --application /private/var/folders/jb/0pl4bj7j5nn_lr3glqvv3l6w0000gn/T/arduino-sketch-80AD89B9246DE12852B2BAA2CFD6DE31/Blink.ino.hex /private/var/folders/jb/0pl4bj7j5nn_lr3glqvv3l6w0000gn/T/arduino-sketch-80AD89B9246DE12852B2BAA2CFD6DE31/Blink.ino.zip

it runs fine. No exception.

Any help would be very much appreciated.

DetErMadsHansen commented 1 year ago

+1 on this.

I'm experiencing the same compile error with an Adafruit Itsy Bitsy nRF52840 Express on Arduino IDE 2.0.2 and MacOS 13.0.1.

But everything is fine on Arduino IDE 1.8.19.

I'm new to both the board, Arduino code/ compiler. So unfortunately I might not be to much help with the debugging.

Library Adafruit nRFCrypto has been declared precompiled:
Using precompiled library in /Users/xyz/Library/Arduino15/packages/adafruit/hardware/nrf52/1.3.0/libraries/Adafruit_nRFCrypto/src/cortex-m4/fpv4-sp-d16-hard
Traceback (most recent call last):
  File "__main__.py", line 317, in <module>
  File "click/core.py", line 1134, in __call__
  File "click/core.py", line 1040, in main
  File "click/_unicodefun.py", line 100, in _verify_python_env
RuntimeError: Click will abort further execution because Python was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/unicode-support/ for mitigation steps.

This system lists some UTF-8 supporting locales that you can pick from. The following suitable locales were discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8
[6025] Failed to execute script __main__

exit status 1

Compilation error: exit status 1
exipilis commented 1 year ago

Exactly the same error on Arduino IDE 2.0.3.

tinxx commented 1 year ago

Same problem here!

Would you please care to remove label conclusion: off topic and maybe add os: macos and/or other labels you deem appropriate?

Thanks!

ApolloDS commented 1 year ago

I think this is a bug which is described here: Unicode/locale error on MacOS and Arduino 2.0.0 #743

panjekm commented 1 year ago

Also having the same issue on my mac with using Arduino 2.0.3 and Adafruit NRF52 board package 1.3. Im faling with the same error when trying to compile for my NRF52832 board.

myst729 commented 1 year ago

any workaround here?

tinxx commented 1 year ago

@myst729 use arduino-cli

zeelodro62 commented 1 year ago

Hello, I am having the same issue shown below. I tried adding following lines uk_UA.UTF-8 export LC_ALL=en_US.UTF-8 but still I get the error.

Traceback (most recent call last):
  File "__main__.py", line 317, in <module>
  File "click/core.py", line 1134, in __call__
  File "click/core.py", line 1040, in main
  File "click/_unicodefun.py", line 100, in _verify_python_env
RuntimeError: Click will abort further execution because Python was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/unicode-support/ for mitigation steps.

This system lists some UTF-8 supporting locales that you can pick from. The following suitable locales were discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8
[3798] Failed to execute script __main__
zeelodro62 commented 1 year ago

??

maxxafari commented 1 year ago

Any updates? I'm stuck at the first blink test. Tried on 2 different M1 Macbooks Will it work if I downgrade to an earlier version of the IDE?

zeelodro62 commented 1 year ago

Yes downgrade to earlier version. That works fine

On Tue, 8 Aug, 2023, 8:19 pm Max Berkenstam, @.***> wrote:

Any updates? I'm stuck at the first blink test. Tried on 2 different M1 Macbooks Will it work if I downgrade to an earlier version of the IDE?

— Reply to this email directly, view it on GitHub https://github.com/arduino/arduino-ide/issues/1513#issuecomment-1670090585, or unsubscribe https://github.com/notifications/unsubscribe-auth/A45O65V6WNB2O7R3EPBKYB3XUJ7LZANCNFSM6AAAAAAQ244EDM . You are receiving this because you commented.Message ID: @.***>