Breaking change from .esphome to /data

[Fluffkin]

The problem

Just to clarify the situation here. There's a breaking change in the latest release https://github.com/esphome/esphome/commit/fe81bcc00343ba8d99a437762087717d45a20946 that moves the .esphome contents to /data.

It's not documented.

It contains code that will delete my existing .esphome folder without asking when I restart and load esphome.

At the moment it's still compiling the updated version apparently using the .esphome folder (?!). There's nothing in /data after test upgrading one of my ESP8266 units.

There's no easy way for the user to install fonts etc to /data because it's not exposed by the Samba server.

When does this time bomb go off exactly and delete all my fonts etc from .esphome without moving them for me first? When I reboot HA?

Which version of ESPHome has the issue?

2023.9.0

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

2023.09.2

What platform are you using?

ESP8266

Board

d1 mini

Component causing the issue

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[its-me-jake]

Following because I just nuked my installation this morning. Similar issue - I cannot find the /data directory at all, and my configuration was all set up using packages. I did a test installation of a new Wemos D1 to see if that helped, but no. All my devices are still working, but I can't update them or figure out where to put my package files as I rebuild them.

[3ative]

Chatting with one of the DEVs this morning. His advice was to shell in to: cd /mnt/data/supervisor/addons/data/5c53de3b_esphome/

And: "if you need to do a lot of debugging with the files in the .esphome directory, you should install it locally". Then he pointed me to this link: https://esphome.io/guides/installing_esphome#windows

Personally, this "Breaking Change" is a real kick in the NADs for us tinkerers and tutorial makers - I'm really annoyed

[ssieb]

I think you're all misunderstanding. You should never have put anything in .esphome. You don't need to access the /data directory either. These are temporary storage areas for esphome to use while compiling. Any files you need to use are in /config/esphome or normal subdirectories under that.. That's where you put your yaml, images, fonts, etc.

[Fluffkin]

Ahh. This maybe where documentation would have been an idea? IIRCC when I first got displays working in esphome I put the fonts in .esphome

Edit: I also seem to remember tinkering with .esphome when I was trying to get the Bosch libraries for a BME688 working.

Yes there does seem to be confusion, but if it doesn't effect anything .. why is it a breaking change? And if it doesn't really change anything.. why are there above comments on "installing .esphome locally" & "I nuked my install this morning"?

[ssieb]

I'm curious why you would do that. . directories are usually "hidden" and reserved for application use. Everything in the docs says to put extra files in the same directory as the yaml files or a directory under it that you create. "installing .esphome locally" is misunderstanding what was said. The comment was that if you need to look at the build files in .esphome for debugging purposes, then you should install esphome locally so that you have easy access to it. But you should never be putting any files in there yourself.

[its-me-jake]

@ssieb how would one implement a package-based configuration with the new version? I followed this: https://github.com/jcallaghan/esphome-config and after I updated, all of my files were just gone. I realize that I have to rebuild everything one way or the other, but I also have another installation that I have not upgraded yet, and I would like to know how to migrate that if it is possible.

[ssieb]

I don't see how anything there would be affected by this change. Maybe come to discord, so it's easier to discuss.

[leoshusar]

I am currently experiencing this issue:

-- Configuring incomplete, errors occurred!

CMake Error at /data/cache/platformio/packages/framework-espidf/tools/cmake/targets.cmake:108 (message):
   Target 'esp32' in sdkconfig '/data/build/4heat-controller/sdkconfig.4heat-controller' does not match currently selected IDF_TARGET 'esp32c6'. To change the target, clear the build directory and sdkconfig file, and build the project again.
Call Stack (most recent call first):
  /data/cache/platformio/packages/framework-espidf/tools/cmake/project.cmake:24 (__target_init)
  CMakeLists.txt:2 (include)

Since the UI "new device" dialog doesn't support C6 yet, I selected ESP32 device type and then wanted to just change the board. But just changing the board apparently doesn't clean the sdkconfig, so I needed to delete it using docker exec in advanced terminal.

IMO there are legitimate use cases for accessing build directory - for example for cleaning things that "clean build files" doesn't clean.

[DjordjeMandic]

Honestly i would like to have access to the data. Sometimes reading generated code helps better understanding it, copying some files for debugging and etc. It's fine for everything to stay in isolated /data but please make a mount/link to .esphome directory for easy access.

[ssieb]

~The clean build command clears everything out for that specific device. There is nothing more that you could do with file access. It literally deletes the entire build directory for that device.~ (Turns out this is wrong and will be fixed soon.) There is no way to link to the /data directory. And as I mentioned before, the only generated file is main.cpp. Everything else you already have easy access to.

[ssieb]

https://github.com/esphome/esphome/pull/5443