Tool Chain Error

[arkansas-coder]

The problem

There appears to be a spelling error in the tool chain. Instead of charset_normalizer it has ~harset_normalizer. See the log smippet below.

Which version of ESPHome has the issue?

2022.12.6

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

No response

What platform are you using?

ESP32

Board

esp32-C3-Dev1

Component causing the issue

No response

Example YAML snippet

esphome:
  name: esp2

esp32:
  board: esp32dev
  variant: esp32C3
  framework:
    type: esp-idf

# Enable logging
logger:

# Enable Home Assistant API
api:
    password: "xxxxxx"

ota:
    password: "xxxxxx"

wifi:
  ssid: "yyyyy"
    password: "xxxxxx"

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp2 Fallback Hotspot"
    password: "xxxxxx"

Anything in the logs that might be useful for us?

Building wheels for collected packages: future
  Building wheel for future (setup.py): started
  Building wheel for future (setup.py): finished with status 'done'
  Created wheel for future: filename=future-0.18.3-py3-none-any.whl size=492055 sha256=fab9217925f929a3c766fe3a672a71c8f0ba24fc9ce260efb01fa1722f596964
  Stored in directory: c:\users\uuuuu\appdata\local\pip\cache\wheels\83\31\5a\7539ed9c6ac3249781de831638629a4fe503168023e1a777e2
Successfully built future
Installing collected packages: pyparsing, msgpack, lockfile, charset-normalizer, urllib3, six, pyyaml, pycparser, packaging, idna, future, contextlib2, colorama, certifi, tqdm, schema, requests, click, cffi, requests-toolbelt, requests-file, cryptography, cachecontrol, idf-component-manager
  Attempting uninstall: charset-normalizer
    Found existing installation: charset-normalizer 3.0.1
    Uninstalling charset-normalizer-3.0.1:
      Successfully uninstalled charset-normalizer-3.0.1
ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Users\\uuuuu\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\~harset_normalizer\\md.cp311-win_amd64.pyd'
Consider using the `--user` option or check the permissions.

*** Error 1

Additional information

This is the first time I am trying to build firmware using ESPHome.

[dfsx1]

I experienced the same error, also with 2022.12.8 Went back to arduino framework (works fine for me):

  framework:
    type: arduino

[arkansas-coder]

Using arduino works. thanks! However, it still appears to be a bug in the chain.

[thr33bricks]

You are not running the "esphome run file_name.yaml" command as admin. Consider opening PowerShell as admin by rightclicking it and select "Run as administrator". Then go to the .yaml configuration location and execute "esphome run file_name.yaml".

[dmmps]

You are not running the "esphome run file_name.yaml" command as admin. Consider opening PowerShell as admin by rightclicking it and select "Run as administrator". Then go to the .yaml configuration location and execute "esphome run file_name.yaml".

Running ps as admin doesn't resolve the problem. Still "Access is denied" error.

[nauticus25]

I am having the same issue. First time trying to build firmware with ESPHome. Almost the same exact error except mine outputs:

ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Users\\uuuuu\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\site-packages\\~~arset_normalizer\\md.cp311-win_amd64.pyd'

I've run as administrator as suggested, but that does not resolve the issue.

[nauticus25]

Interestingly, I have a few folders in the location mentioned in the error. I tried to compile the firmware three times, and now have a ~harset_normalizer folder, a ~-arset_normalizer folder, and a ~~arset_normalizer folder. Each folder was created during one of the attempts, and each folder has several files inside.

[ssieb]

What are you actually running? Compiling the yaml file won't be installing any python packages, so what are you doing?

[nauticus25]

In my case, the command I'm running is esphome run filename.yaml which does appear to install python packages. I'm following the procedure to flash esphome to an Emporia Vue energy monitor as outlined here: https://github.com/emporia-vue-local/esphome.

[playya]

I think I have a similiar issue using latest docker container. To reproduce this one only needs to compile the bt-proxy example using 2012.0 docker image: https://esphome.io/components/bluetooth_proxy.html?highlight=bt+proxy#complete-sample-recommended-configuration

Output:

INFO Reading configuration /config/bt-proxy.yaml... WARNING GPIO12 is a Strapping PIN and should be avoided. Attaching external pullup/down resistors to strapping pins can cause unexpected failures. See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins INFO Generating C++ source... INFO Core config or version changed, cleaning build files... INFO Deleting /config/.esphome/build/my-bluetooth-proxy/.pioenvs INFO Compiling app... Processing my-bluetooth-proxy (board: esp32-poe-iso; framework: espidf; platform: platformio/espressif32 @ 5.2.0)

HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash

• framework-espidf @ 3.40402.0 (4.4.2)
• tool-cmake @ 3.16.4
• tool-ninja @ 1.7.1
• toolchain-esp32ulp @ 2.35.0-20220830
• toolchain-xtensa-esp32 @ 8.4.0+2021r2-patch3 Installing ESP-IDF's Python dependencies Collecting future>=0.15.2 Using cached future-0.18.3-py3-none-any.whl Collecting pyparsing<2.4.0,>=2.0.3 Using cached pyparsing-2.3.1-py2.py3-none-any.whl (61 kB) Collecting idf-component-manager~=1.0 Using cached idf_component_manager-1.2.2-py3-none-any.whl (116 kB) Collecting packaging Using cached packaging-23.0-py3-none-any.whl (42 kB) Collecting requests<3 Using cached requests-2.28.2-py3-none-any.whl (62 kB) Collecting click Using cached click-8.1.3-py3-none-any.whl (96 kB) Collecting requests-file Using cached requests_file-1.5.1-py2.py3-none-any.whl (3.7 kB) Collecting six Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting colorama Using cached colorama-0.4.6-py2.py3-none-any.whl (25 kB) Collecting pyyaml>5.2 Using cached PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB) Collecting contextlib2>0.6.0 Using cached contextlib2-21.6.0-py2.py3-none-any.whl (13 kB) Collecting cachecontrol[filecache]>0.12.6 Using cached CacheControl-0.12.11-py2.py3-none-any.whl (21 kB) Collecting requests-toolbelt Using cached requests_toolbelt-0.10.1-py2.py3-none-any.whl (54 kB) Collecting schema Using cached schema-0.7.5-py2.py3-none-any.whl (17 kB) Collecting tqdm<5 Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB) Collecting msgpack>=0.5.2 Using cached msgpack-1.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (322 kB) Collecting lockfile>=0.9 Using cached lockfile-0.12.2-py2.py3-none-any.whl (13 kB) Collecting urllib3<1.27,>=1.21.1 Using cached urllib3-1.26.14-py2.py3-none-any.whl (140 kB) Collecting charset-normalizer<4,>=2 Using cached charset_normalizer-3.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (198 kB) Collecting certifi>=2017.4.17 Using cached certifi-2022.12.7-py3-none-any.whl (155 kB) Collecting idna<4,>=2.5 Using cached idna-3.4-py3-none-any.whl (61 kB) Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests, msgpack, six, lockfile, contextlib2, cachecontrol, tqdm, schema, requests-toolbelt, requests-file, pyyaml, packaging, future, colorama, click, pyparsing, idf-component-manager Attempting uninstall: idna Found existing installation: idna 3.4 Uninstalling idna-3.4: ERROR: Could not install packages due to an EnvironmentError: [Errno 22] Invalid argument: '/usr/local/lib/python3.9/dist-packages/idna-3.4.dist-info/'

*** Error 1 Reading CMake configuration... -- Found Git: /usr/bin/git (found version "2.30.2") -- The C compiler identification is GNU 8.4.0 -- The CXX compiler identification is GNU 8.4.0 -- The ASM compiler identification is GNU -- Found assembler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch3/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch3/bin/xtensa-esp32-elf-gcc -- Check for working C compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch3/bin/xtensa-esp32-elf-gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch3/bin/xtensa-esp32-elf-g++ -- Check for working CXX compiler: /config/.esphome/platformio/packages/toolchain-xtensa-esp32@8.4.0+2021r2-patch3/bin/xtensa-esp32-elf-g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Building ESP-IDF components for target esp32 -- Configuring incomplete, errors occurred! See also "/config/.esphome/build/my-bluetooth-proxy/.pioenvs/my-bluetooth-proxy/CMakeFiles/CMakeOutput.log".

fatal: not a git repository (or any parent up to mount point /) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). CMake Error at /config/.esphome/platformio/packages/framework-espidf/tools/cmake/build.cmake:454 (message): /usr/bin/python3: Error while finding module specification for 'idf_component_manager.prepare_components' (ModuleNotFoundError: No module named 'idf_component_manager')

Call Stack (most recent call first): /config/.esphome/platformio/packages/framework-espidf/tools/cmake/project.cmake:384 (idf_build_process) CMakeLists.txt:3 (project)

========================= [FAILED] Took 59.57 seconds =========================

[playya]

Same issue on :dev container I think something requests an update of urllib3:

Collecting urllib3<1.27,>=1.21.1

while 1.26.14 is available:

Found existing installation: urllib3 1.26.14

maybe docker container needs a rebuild?

[playya]

I tried to remove charset-normalizer from command line and get this error:

oot@esphome:/config# pip3 uninstall charset-normalizer [194/617] Found existing installation: charset-normalizer 3.0.1 Uninstalling charset-normalizer-3.0.1: Would remove: /usr/local/bin/normalizer /usr/local/lib/python3.9/dist-packages/charset_normalizer-3.0.1.dist-info/ /usr/local/lib/python3.9/dist-packages/charset_normalizer/ Proceed (y/n)? y ERROR: Exception: Traceback (most recent call last): File "/usr/lib/python3.9/shutil.py", line 806, in move os.rename(src, real_dst) OSError: [Errno 18] Invalid cross-device link: '/usr/local/lib/python3.9/dist-packages/charset_normalizer-3.0.1.dist-info/' -> '/usr/local/lib/python3.9/dist-packages/~-arset_normalizer- 3.0.1.dist-info'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 223, in _main status = self.run(options, args) File "/usr/lib/python3/dist-packages/pip/_internal/commands/uninstall.py", line 89, in run uninstall_pathset = req.uninstall( File "/usr/lib/python3/dist-packages/pip/_internal/req/req_install.py", line 694, in uninstall uninstalled_pathset.remove(auto_confirm, verbose) File "/usr/lib/python3/dist-packages/pip/_internal/req/req_uninstall.py", line 403, in remove moved.stash(path) File "/usr/lib/python3/dist-packages/pip/_internal/req/req_uninstall.py", line 292, in stash renames(path, new_path) File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 355, in renames shutil.move(old, new) File "/usr/lib/python3.9/shutil.py", line 818, in move rmtree(src) File "/usr/lib/python3.9/shutil.py", line 722, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/usr/lib/python3.9/shutil.py", line 720, in rmtree os.rmdir(path) OSError: [Errno 22] Invalid argument: '/usr/local/lib/python3.9/dist-packages/charset_normalizer-3.0.1.dist-info/'

This creates the beforemetioned ~ directories. pip does not consider to have charset-normalizer installed, but issue is not fixed

[ssieb]

There is clearly something wrong with your Python environment. Try clearing everything out and try again.

[flaviut]

Someone's running into this here too, on Windows: https://github.com/emporia-vue-local/esphome/discussions/119

They've tried:

• reinstalling python
• running from a virtualenv

I suspect this is some Windows path idiosyncrasy, since ~ is often a special character there.

@nauticus25 what exact OS version are you running, and what's the exact python installer you're using?

[marshn]

Hey @playya, is this similar to https://github.com/esphome/issues/issues/3961 ?

Try running this command and see if anything changes:

pip install -U --force-reinstall "idf-component-manager~=1.0"

[nauticus25]

Someone's running into this here too, on Windows: emporia-vue-local/esphome#119

They've tried:

* reinstalling python

* running from a virtualenv

I suspect this is some Windows path idiosyncrasy, since ~ is often a special character there.

@nauticus25 what exact OS version are you running, and what's the exact python installer you're using?

I'm running Windows 10 Home 21H2 and Python 3.11.1.

[playya]

Hey @playya, is this similar to #3961 ?

Try running this command and see if anything changes:

pip install -U --force-reinstall "idf-component-manager~=1.0"

This fails with the same error. I'm using official docker container using overlay2 storage driver. While googling on this issue I found some evidence that the renaming is failing because of the overlays are seen as different filesystems.

Im now back at know working version 2022.10.2, but it also broken. I have to revise my docker config.

[playya]

I need to double check again. It seems to be broken, again

WRONG: I reconfigured to use btrfs storage driver and I'm able to install the python packages.

[flaviut]

@nauticus25 not sure what's wrong, I can't reproduce this. Works fine for me on Windows 10 22H2. Can you try another computer or an alternative installation method?

[RickSisco]

Someone's running into this here too, on Windows: emporia-vue-local/esphome#119 They've tried:

* reinstalling python

* running from a virtualenv

I suspect this is some Windows path idiosyncrasy, since ~ is often a special character there. @nauticus25 what exact OS version are you running, and what's the exact python installer you're using?

I'm running Windows 10 Home 21H2 and Python 3.11.1.

Any luck resolving this issue?

[nauticus25]

Someone's running into this here too, on Windows: emporia-vue-local/esphome#119 They've tried:

* reinstalling python

* running from a virtualenv

I suspect this is some Windows path idiosyncrasy, since ~ is often a special character there. @nauticus25 what exact OS version are you running, and what's the exact python installer you're using?

I'm running Windows 10 Home 21H2 and Python 3.11.1.

Any luck resolving this issue?

Not yet. I downgraded to Python 3.10.10, but got the same errors. My next attempt will be on a different computer in the next few days. I’ll post back with the results.

[RickSisco]

I tried updating a power monitoring system with esphome before I came to work and had a similar, of not the same, issue. Was looking it up on break, but will look at it a little when I get home a bit past 1am EST to see if i can figure it out. I'll report back if I figure it out as well.

On Mon, Feb 13, 2023, 8:34 PM nauticus25 @.***> wrote:

Someone's running into this here too, on Windows: emporia-vue-local/esphome#119 https://github.com/emporia-vue-local/esphome/discussions/119 They've tried:

• reinstalling python

• running from a virtualenv

I suspect this is some Windows path idiosyncrasy, since ~ is often a special character there. @nauticus25 https://github.com/nauticus25 what exact OS version are you running, and what's the exact python installer you're using?

I'm running Windows 10 Home 21H2 and Python 3.11.1.

Any luck resolving this issue?

Not yet. I downgraded to Python 3.10.10, but got the same errors. My next attempt will be on a different computer in the next few days. I’ll post back with the results.

— Reply to this email directly, view it on GitHub https://github.com/esphome/issues/issues/4084#issuecomment-1428974773, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABYCTSYMOSOMXLO2NRNJRSDWXLOLPANCNFSM6AAAAAAUH77IHM . You are receiving this because you commented.Message ID: @.***>

[RickSisco]

Just as a followup, I has the same exact error.. "ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\Users\BillGatesSucks\AppData\Local\Programs\Python\Python311\Lib\site-packages\~harset_normalizer\md.cp311-win_amd64.pyd'"

I tried several things to get it to work. I gave up and just tried installing everything on my laptop and received the same error. At this point I had one last option, I took a spare ssd and used a usb to ssd adapter, loaded the generic x86 Home Assistant to it with balenaEtcher. Then swapped that drive into my laptop. I booted up, set up Home Assistant, installed ESPHome, added a new device, used the contents of my vue2.yaml and saved. Installed from there and it went fine. Shut down the laptop and reinstalled my original harddrive. Opened up the home assistant page for the raspberry pi it runs on and the emporia vue 2 was online and working. :) -- Note: I couldn't install it from the pi as it errored out, likely due to lack of memory.

[nauticus25]

I just tried on a fresh install on a different computer. Same error.

So, I read up a ways in this thread and saw the suggestion made to another user to run pip install -U --force-reinstall "idf-component-manager~=1.0". So I did. Then I ran esphome run emporiavue.yaml again and it worked! I looked back at the logs and the idf-component-manager forced reinstall command goes through the same "uninstall charset-normalizer" routine, but this time it successfully completes the uninstall and reinstall. Then running the esphome run emporiavue.yaml command does not attempt to uninstall/reinstall charset-normalizer (or most of the other packages it had been attempting to previously), and it successfully compiles the firmware.

tl;dr - pip install -U --force-reinstall "idf-component-manager~=1.0" fixed the problem.

Thanks!

[sjschultze]

Thanks to @nauticus25 for posting this and highlighting the solution. I had the exact same problem and this fixed it.

[ericgkr]

Thank you @nauticus25 . I confirm that your solution works for me too! I hit exactly the same issue on Win11 (Python 3.11).