utf-8' codec can't decode byte 0xb0 in position 37

dummylabs / thewatchman

Home Assistant custom integration to keep track of missing entities and actions in your config files

MIT License

480 stars 20 forks source link

utf-8' codec can't decode byte 0xb0 in position 37 #138

Closed Mariusthvdb closed 4 months ago

Mariusthvdb commented 5 months ago

This is happening while running the HA beta 2024.7.0b3 and the Main fix for the raised blocking call which I downloaded and restarted just now

Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.watchman.utils
Bron: custom_components/watchman/utils.py:291
integratie: Watchman (documentatie, problemen)
Eerst voorgekomen: 13:24:03 (15 gebeurtenissen)
Laatst gelogd: 13:24:05

Unable to parse /config/._device_tracker.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error
Unable to parse /config/._recorder.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error
Unable to parse /config/._customizer.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error
Unable to parse /config/._frontend.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error
Unable to parse /config/custom_components/._services.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error

dummylabs commented 5 months ago

Thanks for the report. It looks a bit weird that all these errors refer to the same position in different files. BTW, all files are hidden (as they name starts from '.'), i.e. they will not be picked up by Home Assistant. Are you sure those have UTF-8 format and have no issues from Home Assistant side? ~Anyway it's a good catch, we need to add an additional check here~

Can you confirm that this error doesn't occur when running W 0.6.2 against HA 2024.7.*?
Can you please provide one of those yaml files (without personal information) for analysis?

Update: I've reviewed the code, watchman catches this exception and reports error to the log file as it should. Most probably your files are not valid UTF-8 files. As a solution one may open a file in an editor which support encoding selection and save it as UTF-8 again. Or you may remove those files (or change their .yaml extension) to exclude them from Watchman parsing.

Mariusthvdb commented 4 months ago

tbh, I dont have a clue what you mean when you mention 'one of those yaml files'?

I did not see the errors being logged after another restarts maybe it was a one time fluke?

edit

wait, silly me. let me check I have none of those mentioned hidden files....

Mariusthvdb commented 4 months ago

cant this be caused by the changes that were recently idoen? https://github.com/dummylabs/thewatchman/commit/18b5d688f12d3c3f3c2d00e1cf10c0c6acb3c561

again, the hidden files mentioned here:

are not on my system ( Ive enabled hidden file viewing in Mac finder on the /config, so they should show I SMB share if existing

dummylabs commented 4 months ago

cant this be caused by the changes that were recently idoen?

I don't think so. It can be that during installation of new HA beta some migration step took place which made these temporary files and Watchman picked them up automatically. I'll consider adding an additional filename filter to skip hidden files from parsing as they normally are not used by HA. Thank you for the feedback, I'll close the ticket, feel free to reopen it if necessary.

Mariusthvdb commented 4 months ago

Wait, I decided to check those file names with Studio Code Server in HA directly, and not the file manager on the Mac over Samba.

it appears these files Do exist:

no clue yet what they do or when/why created but at least there’s is some progress

If these files are legit and always there it would indeed be nice to be able to excluded them from the configuration with an option.

Mariusthvdb commented 4 months ago

now what I believe happened, is that my default editor BBEdit creates those files upon opening/editing, and doesnt remove them when ready/saving/closing those files.

Also, my SMB add-on hid those files by vetoing them, so I couldn't see them in the Finder, but the VSCode add-on was not filtering these files.

Remains the why: why did this start to happen lately, I've been using BBEdit since forever.... an update was installed, maybe some settings changed, although I checked and couldn't really find an obvious option to control that.

Right now, I've manually deleted all those files, and set Watchman to ignore ._* and hope that is the right setting.

Otoh, in hindsight, it might be a good idea to have some checker keep an eye out for those hidden file creations/remnants

edit

apparently not the correct filter....

Unable to parse /config/dashboard/buttons/._buttons_schakelaars.yaml: 'utf-8' codec can't decode byte 0xb0 in position 37: invalid start byte. Use UTF-8 encoding to avoid this error