[Bug] Unexpected char '<' at line 1, column 1 (JSON::ParseException) | 429 rate limit errors

[corincorvus]

UPDATE from @unixfox:

See here for solutions: https://docs.invidious.io/youtube-errors-explained/

---

Describe the bug just want to watch a video

Steps to Reproduce

Logs

Title: `Unexpected char '<' at line 1, column 1 (JSON::ParseException)`
Date: `2024-10-29T15:20:09Z`
Route: `/watch?v=Q314YvZvnDY`
Version: `2024.08.26-4782a67 @ (HEAD detached at v2.20240825.2)`

<details>
<summary>Backtrace</summary>
<p>

Unexpected char '<' at line 1, column 1 (JSON::ParseException)
  from /usr/share/crystal/src/json/lexer.cr:331:5 in 'raise'
  from /usr/share/crystal/src/json/lexer.cr:327:5 in 'unexpected_char'
  from /usr/share/crystal/src/json/lexer.cr:326:11 in 'unexpected_char'
  from /usr/share/crystal/src/json/lexer/io_based.cr:11:21 in 'consume_number'
  from /usr/share/crystal/src/json/lexer.cr:63:7 in 'next_token'
  from /usr/share/crystal/src/json/parser.cr:13:12 in '_post_json'
  from src/invidious/videos/parser.cr:58:21 in 'extract_video_info'
  from src/invidious/videos.cr:381:10 in 'fetch_video'
  from src/invidious/videos.cr:369:13 in 'get_video:region'
  from src/invidious/routes/watch.cr:55:15 in 'handle'
  from lib/kemal/src/kemal/route.cr:13:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:106:14 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:141:11 in 'run'
  from ???
</p>
</details>

Log in some containers: ``` 2024-10-29 15:17:41 UTC [info] 500 GET /api/v1/comments/jNQXAC9IVRw 1241.61ms 2024-10-29 15:18:07 UTC [info] SigHelper: Using helper at 'inv_sig_helper:12999' [production] Invidious is ready to lead at http://0.0.0.0:3000 2024-10-29 15:18:07 UTC [info] jobs: running ClearExpiredItems job 2024-10-29 15:18:07 UTC [info] jobs: ClearExpiredItems done. 2024-10-29 15:18:07 UTC [error] RefreshChannelsJob: UCUQl1JLdletOp5Bi3tWojxA : Unexpected char '<' at line 1, column 1 2024-10-29 15:18:07 UTC [error] RefreshChannelsJob: UCUQl1JLdletOp5Bi3tWojxA fiber : backing off for 00:02:00s 2024-10-29 15:18:07 UTC [error] RefreshChannelsJob: UCdjedrfgyFQEkqbkq1DJf3w : Unexpected char '<' at line 1, column 1 2024-10-29 15:18:07 UTC [error] RefreshChannelsJob: UCdjedrfgyFQEkqbkq1DJf3w fiber : backing off for 00:02:00s 2024-10-29 15:18:15 UTC [info] 200 POST /feed/webhook/v1:1729947710:8fd1e432:8b70b21d3f319072bc3ad18322df815edc720497 363.89µs 2024-10-29 15:19:10 UTC [info] 500 GET /api/v1/comments/jNQXAC9IVRw 3374.61ms 2024-10-29 15:20:08 UTC [error] RefreshChannelsJob: UCJpmsAafudni1k6RQo0-gjQ : Unexpected char '<' at line 1, column 1 2024-10-29 15:20:08 UTC [error] RefreshChannelsJob: UCJpmsAafudni1k6RQo0-gjQ fiber : backing off for 00:08:00s 2024-10-29 15:20:08 UTC [error] RefreshChannelsJob: UCv1WDP5EiipMQ__C4Cg6aow : Unexpected char '<' at line 1, column 1 2024-10-29 15:20:08 UTC [error] RefreshChannelsJob: UCv1WDP5EiipMQ__C4Cg6aow fiber : backing off for 00:08:00s 2024-10-29 15:20:11 UTC [info] 500 GET /api/v1/comments/jNQXAC9IVRw 492.39ms 2024-10-29 15:21:11 UTC [info] 500 GET /api/v1/comments/jNQXAC9IVRw 470.33ms 2024-10-29 15:22:12 UTC [info] 500 GET /api/v1/comments/jNQXAC9IVRw 570.25ms 2024-10-29 15:22:17 UTC [info] 200 POST /feed/webhook/v1:1729947709:32070908:af18766e41ecb4f132c5fbfc0cb4d842080b9da9 258.02µs 2024-10-29 15:22:19 UTC [info] 200 POST /feed/webhook/v1:1729947710:5a8637c3:6ac8a425792bac21a5a7b94e4f56a00a594536d3 247.4µs ``` **Screenshots**   

Additional context Server is Debian and i have many Containers for Indivious Its private Hosting Latest Containers Postgres Version 14, like in the Compose.

I already tried

• a restore
• a restart
• new pull images and redeploy
• test without loggged in user
• test wirh logged in user

[yatzy]

I started to get the same error for a previously working installation within the last few hours

[unixfox]

Ok interesting. @yatzy is https://www.youtube.com working for you?

[scoliono]

Seconding this issue. This only started happening on my instance this morning when trying to load the Trending page, watch videos, etc. My fork only has a few small modifications, essentially the same as Jing's instance, which still works. edit: the feed is probably just cached, videos not loading anymore.

https://git.linux.ucla.edu/lug/invidious

I was able to download a YouTube video using yt-dlp on the same machine, and I was able to open the homepage in Lynx, so I don't believe I am blocked.

[yatzy]

Out of my heart cannot remember if i'm normally seeing this cookie screen which I was forced to accept

Prior accepting the cookies I got this when browsing to the site from the Youtube's side panel.

After accepting the cookies Youtube started working. My Invidious instance is also working again. Don't know what's going on. Hope this helps!

EDIT: My instance did work for an hour or so, and now I'm again facing the Title: Unexpected char '<' at line 1, column 1 (JSON::ParseException) error

EDIT 2: Ok it seems I'm just plain blocked, with YT app on my phone on wifi I get There was a problem with a server, but on mobile it works fine

[radiowavesurfer]

Similarly, I wasn't seeing this "unexpected char" error 5 hours ago when this issue was opened, but I'm now seeing it on every video.

[unixfox]

Related issues:

• https://github.com/TeamNewPipe/NewPipe/issues/11661
• https://github.com/FreeTubeApp/FreeTube/issues/6020
• https://old.reddit.com/r/youtube/comments/1gev7jz/anyone_getting_429_cant_do_anything_in_android_app/

[unixfox]

Is everyone here already using po_token method?

[radiowavesurfer]

Is everyone here already using po_token method?

I wasn't when the problem first arose, but have now upgraded to the current release, got the po_token (which seemed to go ok), but still seeing the "Unexpected char" error. I'm going to leave invidious shut down for a few days, then give it another try.

[unixfox]

Does it always happen after watching a video or the error message always happen regardless of what you do?

Did you try to turn off invidious for a few hours just to see if that temporarily unblock you?

[NightSlayer-007]

I think there's a chance this could be related to something seen in Reddit clones recently as well with the suspected change to http2: https://github.com/redlib-org/redlib/issues/301

[corincorvus]

Does it always happen after watching a video or the error message always happen regardless of what you do?

Did you try to turn off invidious for a few hours just to see if that temporarily unblock you?

I get the error by click on a video to watch on my abo list.

I need to reconnect my internet. 6 hours stop indivious didnt unblock me.

[mariowo]

This is just a suspicion, but in my observation (using locally-hosted Invidious as a backend for FreeTube), the rate-limiting only occurs after fetching my subscriptions. Using a non-limited IP apparently gives me unlimited access to view as many videos as I please, but as soon as I refresh my subscriptions (using RSS), I get the 429 error when trying to get a video stream on the now-limited IP. The issue is resolved when using any means to get a non-limited IP address.

[pimlie]

This is just a suspicion, but in my observation (using locally-hosted Invidious as a backend for FreeTube), the rate-limiting only occurs after fetching my subscriptions.

If that is the case then the first full paragraph of this comment might be relevant: https://github.com/TeamNewPipe/NewPipe/issues/11661#issuecomment-2448344651

Not sure if invidious also has a 'fast-mode' to retrieve subscriptions like NewPipe has?

[mariowo]

Interestingly enough, I am able to watch videos on NewPipe (actually Tubular) with a rate-limited IP (by fetching the subscriptions through FreeTube with Invidious Backend, but not through NewPipe), while my PC gets a [BAD_HTTP_STATUS: 403] YouTube watch session expired. Please reopen this video. when trying to watch them through FreeTube with the Invidious Backend using the same IP.
It seems to me that there might also be some kind of User-Agent or Hardware fingerprint identification happening to check for the Rate-Limiting?

[unixfox]

Interestingly enough, I am able to watch videos on NewPipe (actually Tubular) with a rate-limited IP (by fetching the subscriptions through FreeTube with Invidious Backend, but not through NewPipe), while my PC gets a [BAD_HTTP_STATUS: 403] YouTube watch session expired. Please reopen this video. when trying to watch them through FreeTube with the Invidious Backend using the same IP.
It seems to me that there might also be some kind of User-Agent or Hardware fingerprint identification happening to check for the Rate-Limiting?

Could be that newpipe use the ipv6 network and invidious ipv4 one if you installed it with docker

[mariowo]

Could be that newpipe use the ipv6 network and invidious ipv4 one if you installed it with docker

My Invidious instance is running inside docker, so that is very likely. A quick IP lookup from my phone in fact verifies this.
Sorry for the false lead, my phone uses a different IP to connect to YouTube than my local Invidious.

[Exw27]

From my testing on a local instance I figured that every time I use Materialus with my instance I get rate limited and the Unexpected char '<' at line 1, while using only Invidious on it sown I don't get rate limited. The rate limiting lasts around 24 hours.

[unixfox]

Closing as now this error is documented with solutions: https://docs.invidious.io/youtube-errors-explained/

From this PR: https://github.com/iv-org/documentation/pull/612