Open Spenz0r opened 1 month ago
Similar issue here. The new version fails for every URL for me. The error message is slightly different though. It says:
'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
My environment:
OS: Windows 11 x64
Python: 3.12.4
Installation: pip install changedetection.io
Detailed log (URL redacted)
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID dd17e2da-ab10-4868-9128-74c44f1d9571 Priority 1720743143 URL https://1
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID ff696b8a-2c49-4003-8f97-d416d87d94de Priority 1720743143 URL https://2
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID e48eaabf-fc04-4b42-8e8f-da7d60bf7e79 Priority 1720743143 URL https://3
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 47396e9b-1773-4c53-ba96-fc971a29ad9a Priority 1720743143 URL https://4
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 9eefea13-9966-45fa-b60f-136af0170359 Priority 1720743143 URL https://5
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 11f7ae73-53ac-4cdf-9ad3-67406c2b721e Priority 1720743143 URL https://6
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 68950add-0527-482a-8840-6f8d5b54fa3b Priority 1720743143 URL https://7
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 8252c3b7-f659-4488-8609-743bea2d1eaf Priority 1720743143 URL https://8
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 1b5debb2-cde4-4b00-b129-42a4380330bc Priority 1720743143 URL https://9
2024-07-11 17:12:27.938 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 3690b4bd-d8b5-430b-bb3d-34a2a9aaf38e Priority 1720743143 URL https://10
2024-07-11 17:12:27.938 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: dd17e2da-ab10-4868-9128-74c44f1d9571
2024-07-11 17:12:27.938 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: ff696b8a-2c49-4003-8f97-d416d87d94de
2024-07-11 17:12:27.938 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 9eefea13-9966-45fa-b60f-136af0170359
2024-07-11 17:12:27.938 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: e48eaabf-fc04-4b42-8e8f-da7d60bf7e79
2024-07-11 17:12:27.938 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 47396e9b-1773-4c53-ba96-fc971a29ad9a
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 11f7ae73-53ac-4cdf-9ad3-67406c2b721e
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 68950add-0527-482a-8840-6f8d5b54fa3b
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch dd17e2da-ab10-4868-9128-74c44f1d9571 done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 8252c3b7-f659-4488-8609-743bea2d1eaf
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch ff696b8a-2c49-4003-8f97-d416d87d94de done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 3690b4bd-d8b5-430b-bb3d-34a2a9aaf38e
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 1b5debb2-cde4-4b00-b129-42a4380330bc
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 9eefea13-9966-45fa-b60f-136af0170359 done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch e48eaabf-fc04-4b42-8e8f-da7d60bf7e79 done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 47396e9b-1773-4c53-ba96-fc971a29ad9a done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 11f7ae73-53ac-4cdf-9ad3-67406c2b721e done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 68950add-0527-482a-8840-6f8d5b54fa3b done in 0.02s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 8252c3b7-f659-4488-8609-743bea2d1eaf done in 0.00s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 3690b4bd-d8b5-430b-bb3d-34a2a9aaf38e done in 0.00s
2024-07-11 17:12:27.955 | ERROR | changedetectionio.update_worker:run:477 - 'gbk' codec can't decode byte 0x8c in position 2900: illegal multibyte sequence
2024-07-11 17:12:27.955 | DEBUG | changedetectionio.update_worker:run:569 - Watch 1b5debb2-cde4-4b00-b129-42a4380330bc done in 0.00s
Hmm both sites worked fine for me, but i'm on linux/docker, so I'm thinking this might be a windows related issue
both comments above from windows users
Similar here:
2024-07-12 16:22:44.552 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 23534ea1-8cc3-4c5e-b8c4-53a54b267585
2024-07-12 16:22:44.553 | ERROR | changedetectionio.update_worker:run:477 - 'charmap' codec can't decode byte 0x88 in position 438: character maps to
@luziol "Similar here:" and care to provide any more information? Mac-OS? rPI? windows 3.11?
Windows 10, python only. No docker.
@luziol "Similar here:" care to provide a URL that you used?
Of course:
2024-07-12 17:36:03.387 | INFO | changedetectionio.update_worker:run:255 - Processing watch UUID 23534ea1-8cc3-4c5e-b8c4-53a54b267585 Priority 1 URL https://www.freebsd.org/
2024-07-12 17:36:03.392 | ERROR | changedetectionio.update_worker:run:476 - Exception reached processing watch UUID: 23534ea1-8cc3-4c5e-b8c4-53a54b267585
2024-07-12 17:36:03.393 | ERROR | changedetectionio.update_worker:run:477 - 'charmap' codec can't decode byte 0x88 in position 438: character maps to
Same for me aswell
Exception: 'ascii' codec can't decode byte 0xd8 in position 409: ordinal not in range(128)
with this url
https://api.github.com/repos/dgtlmoon/changedetection.io/releases/latest?q=prerelease%3Afalse+draft%3Afalse&expanded=true
And i apply this filter jqraw:"# [Lien GitHub](" + .html_url + ")\n# " + .tag_name + "\n" + .body
I'm using the docker version in the version v0.46.00
@Erwan-loot and windows or?
PLEASE IF YOU ADD TO THIS THREAD TELL US WHAT OPERATING SYSTEM YOU ARE RUNNING CHANGEDETECTION ON
@Erwan-loot and windows or?
I have edited my post, sorry i'm using the docker version You mean the os used for watch change-detection or the os used to host change detection ?
@Erwan-loot and windows or?
I have edited my post, sorry i'm using the docker version You mean the os used for watch change-detection or the os used to host change detection ?
used for hosting, where it runs
@Erwan-loot and windows or?
I have edited my post, sorry i'm using the docker version You mean the os used for watch change-detection or the os used to host change detection ?
used for hosting, where it runs
So yeah my first answer, docker version here is my docker-compose, normally i use the linuxserver image, i've tested with the original image of course, same result
version: "2.1"
services:
changedetection:
#image: ghcr.io/linuxserver/changedetection.io:latest
image: ghcr.io/dgtlmoon/changedetection.io
container_name: changedetection
environment:
PUID: 1036
PGID: 100
TZ: Europe/Paris
BASE_URL: MYURL
volumes:
#- /myvolume/changedetection:/config
- /myvolume/changedetection:/datastore
ports:
- 7756:5000
restart: unless-stopped
healthcheck:
test: wget --no-verbose --tries=3 --spider http://localhost:5000
security_opt:
- no-new-privileges=true
mem_limit: 256m
This answer helped me: https://stackoverflow.com/a/77048834/1464577
docker run
-d
--name='changedetection.io'
--net='bridge'
-e TZ="America/New_York"
-e HOST_OS="Unraid"
-e HOST_HOSTNAME="Tower"
-e HOST_CONTAINERNAME="changedetection.io"
-e 'BASE_URL'=''
-e 'PLAYWRIGHT_DRIVER_URL'=''
-e 'LC_ALL'='en_US.UTF-8'
-e 'PUID'='99'
-e 'PGID'='100'
-e 'UMASK'='022'
-l net.unraid.docker.managed=dockerman
-l net.unraid.docker.webui='http://[IP]:[PORT:5000]'
-l net.unraid.docker.icon='https://raw.githubusercontent.com/linuxserver/docker-templates/master/linuxserver.io/img/linuxserver-ls-logo.png'
-p '50931:5000/tcp'
-v '/mnt/user/appdata/changedetection.io':'/config':'rw' 'lscr.io/linuxserver/changedetection.io'
Although in my case, I only had to add -e 'LC_ALL'='en_US.UTF-8'
environment variable. (using Unraid)
It seems to fix my issue thanks @jbouwens here the fix in docker-compose
version: "2.1"
services:
changedetection:
image: ghcr.io/linuxserver/changedetection.io:latest
container_name: changedetection
environment:
PUID: 1036
PGID: 100
TZ: Europe/Paris
BASE_URL: MYURL
LC_ALL: en_US.UTF-8
volumes:
- /myvolume/changedetection:/config
ports:
- 7756:5000
restart: unless-stopped
healthcheck:
test: wget --no-verbose --tries=3 --spider http://localhost:5000
security_opt:
- no-new-privileges=true
mem_limit: 256m
This answer helped me: https://stackoverflow.com/a/77048834/1464577
docker run -d --name='changedetection.io' --net='bridge' -e TZ="America/New_York" -e HOST_OS="Unraid" -e HOST_HOSTNAME="Tower" -e HOST_CONTAINERNAME="changedetection.io" -e 'BASE_URL'='' -e 'PLAYWRIGHT_DRIVER_URL'='' -e 'LC_ALL'='en_US.UTF-8' -e 'PUID'='99' -e 'PGID'='100' -e 'UMASK'='022' -l net.unraid.docker.managed=dockerman -l net.unraid.docker.webui='http://[IP]:[PORT:5000]' -l net.unraid.docker.icon='https://raw.githubusercontent.com/linuxserver/docker-templates/master/linuxserver.io/img/linuxserver-ls-logo.png' -p '50931:5000/tcp' -v '/mnt/user/appdata/changedetection.io':'/config':'rw' 'lscr.io/linuxserver/changedetection.io'
Although in my case, I only had to add
-e 'LC_ALL'='en_US.UTF-8'
environment variable. (using Unraid)
Also encountered this issue using the Docker version on Unraid OS. This solution works.
I was running into a similar issue on UnRAID 6.12.11 (Linux 6.1.99 Kernel), using Docker. I am, however, using the linuxserver.io container. The error was as follows:
Exception: 'ascii' codec can't decode byte 0xd8 in position 409: ordinal not in range(128)
This was happening on both "basic" HTTP checks and those using Playwright.
I was able to fix it by adding the 'LC_ALL'='en_US.UTF-8'
environment variable as discussed above. This fixed it for basic HTTP scrapes, but it kept occurring with Playwright powered alerts. Adding the same environment variable to my browserless container fixed it.
I only started running into this issue after updating to the latest version (0.46.01). I was previously two or three versions behind, I think (maybe 0.45.26?).
Perhaps this is something that can be "baked in" to the container (since everything is UTF-8 nowadays), or maybe it is a problem on the linuxserver.io side?
I was running into a similar issue on UnRAID 6.12.11 (Linux 6.1.99 Kernel), using Docker. I am, however, using the linuxserver.io container. The error was as follows:
Exception: 'ascii' codec can't decode byte 0xd8 in position 409: ordinal not in range(128)
This was happening on both "basic" HTTP checks and those using Playwright.
I was able to fix it by adding the
'LC_ALL'='en_US.UTF-8'
environment variable as discussed above. This fixed it for basic HTTP scrapes, but it kept occurring with Playwright powered alerts. Adding the same environment variable to my browserless container fixed it.I only started running into this issue after updating to the latest version (0.46.01). I was previously two or three versions behind, I think (maybe 0.45.26?).
Perhaps this is something that can be "baked in" to the container (since everything is UTF-8 nowadays), or maybe it is a problem on the linuxserver.io side?
That’s exactly the same issue I had encountered. I’m running a recently-updated linuxserver.io image on Unraid 6.12.10, but I don’t have Playwright installed. There was no problem when using ChangeDetecion.io 0.45.26.
Same issues on the latest version.
Windows Server 2019 (v1809). Every URL fails, including the built-in https://changedetection.io/CHANGELOG.txt
They all fail with the error Exception: 'charmap' codec can't decode byte 0x81 in position 440: character maps to <undefined>
Python pip install, no docker. I tried downgrading to version 0.45.26, but it made no difference.
@dgtlmoon I see you've got commit #2515 indicating that it will close this issue, however it only addresses docker containers. This is affecting bare python/pip installs too.
One thing to note with locales on Windows is that they usually use a hyphen instead of an underscore, so _enUS on Linux is en-US on Windows, etc.
On Windows (Server 2019, per my previous comment), no docker, changing flask_app.py:90 to specify the locale to be 'en-US.UTF-8' resolves the warning on launch that the locale can't be set Before:
INFO | changedetectionio.flask_app:<module>:88 - System locale default is ('en_US', 'cp1252')
WARNING | changedetectionio.flask_app:<module>:93 - Unable to set locale ('en_US', 'cp1252'), locale is not installed maybe?
After:
INFO | changedetectionio.flask_app:<module>:88 - System locale default is ('en_US', 'cp1252')
INFO | changedetectionio.store:__init__:48 - Datastore path is
However, it still fails: ERROR | changedetectionio.update_worker:run:479 - 'charmap' codec can't decode byte 0x81 in position 440: character maps to <undefined>
I haven't had much time to look, but I had enough time to try that quick change, so I'm not sure if maybe the locale just needs to be set somewhere else / set as an environment variable. But in any case, I hope this helps.
Hi. I just installed changedetection in Windows 10 Pro, no docker, following your instructions.
I also get the same error:
"Exception: 'charmap' codec can't decode byte 0x81 in position 440: character maps to
Same problem, using the changedetection.io docker on Unraid . Am now using latest versions (app docker 24.05.31 by linuxserver, and changedetection.io v0.46.02)
I have not applied workaround 'LC_ALL'='en_US.UTF-8'
since I read (in this issue) the issue remains when using playwright.
I first time installed this project and I am facing the same problem on windows 10. I used the pip install method.
Same problem, installed via pip method on windows 2019 std server, all sites have the red "Exception: 'charmap' codec can't decode byte 0x81 in position 440: character maps to
Windows 11 and installed via PIP (no docker) with the same error. v0.46.02
Exception: 'charmap' codec can't decode byte 0x81 in position 440: character maps to <undefined>
Adding the encoding="UTF-8"
to line 69 in base.py
resolved the issue.
self.instock_data_js = importlib.resources.files("changedetectionio.content_fetchers.res").joinpath('stock-not-in-stock.js').read_text(encoding="UTF-8")
Additionally, I resolved the Unable to set locale
issue by setting default_locale = locale.getdefaultlocale()
in flask_app.py
Windows 11 and installed via PIP (no docker) with the same error. v0.46.02
Exception: 'charmap' codec can't decode byte 0x81 in position 440: character maps to <undefined>
Adding the
encoding="UTF-8"
to line 69 inbase.py
resolved the issue.self.instock_data_js = importlib.resources.files("changedetectionio.content_fetchers.res").joinpath('stock-not-in-stock.js').read_text(encoding="UTF-8")
Additionally, I resolved the
Unable to set locale
issue by settingdefault_locale = locale.getdefaultlocale()
in flask_app.py
This resolved my issue as well on Windows 11
Hello everyone,
I encountered an issue while deploying this container but was able to resolve it with the help of this and other threads. I wanted to share my experience and solution.
I removed everything unnecessary from my docker-compose.yml file and tested with the configuration below using the default monitored sites.
Operating System: openSUSE Leap 15.6 Docker version: 24.0.7-ce, build 311b9ff0aa93 Docker Compose version: 2.22.0 Container Tag: latest == 0.46.02 (2024-08-15 10:00 2024 UTC)
On the WebUI:
https://changedetection.io/CHANGELOG.txt
Exception: 'ascii' codec can't decode byte 0xd8 in position 409: ordinal not in range(128)
In the Docker logs:
changeio-webserver-1 | 2024-08-15 12:04:12.402 | ERROR | changedetectionio.update_worker:run:478 - Exception reached processing watch UUID: 5c0d902d-8988-4bca-b8d1-f4f0478d3e40
changeio-webserver-1 | 2024-08-15 12:04:12.403 | ERROR | changedetectionio.update_worker:run:479 - 'ascii' codec can't decode byte 0xd8 in position 409: ordinal not in range(128)
---
services:
webserver:
environment:
TZ: 'Europe/Berlin'
image: 'lscr.io/linuxserver/changedetection.io:latest'
ports:
- '10190:5000/tcp'
restart: 'unless-stopped'
---
services:
webserver:
environment:
LC_ALL: en_US.UTF-8
TZ: 'Europe/Berlin'
image: 'lscr.io/linuxserver/changedetection.io:latest'
ports:
- '10190:5000/tcp'
restart: 'unless-stopped'
---
services:
webserver:
environment:
TZ: 'Europe/Berlin'
image: 'lscr.io/linuxserver/changedetection.io:0.45.26'
ports:
- '10190:5000/tcp'
restart: 'unless-stopped'
I also tried version 0.46.00, but this still gave me the error.
Best Regards Alex
does this PR which is now in 0.46.03
help ? https://github.com/dgtlmoon/changedetection.io/pull/2574
@dgtlmoon Yes, the issue has been fixed in 0.46.3
. Thank you!
@jiangzhenjerry fantastic!
Can anyone else confirm the fix?
Yes, confirmed the latest version resolves the problem for me using it directly on Windows 10 with Python
Describe the bug All detection watches I make change to red text with "Exception: charmap codec can t decode byte 0x81 in position 440: character maps to" below their title once they check. I've made them for multiple sites including eBay and they all do this.
Version Latest - 0.45.26
To Reproduce
Steps to reproduce the behavior:
Example URLs: https://www.ebay.com/sch/i.html?_from=R40&_nkw=nintendo+switch+lite&_sacat=0&_udlo=889&_odkw=nintendo+switch+lite&_osacat=0&_sop=10
https://dallas.craigslist.org/search/sss?query=laptop#search=1~gallery~0~0
Desktop (please complete the following information):
Additional context I'm sorry if I'm just reporting something benign, but I worry that it could be making the check not work or causing other issues.