[Bug] Missing hash key: "videoDetails" (KeyError)

[anonas1]

Title: Missing hash key: "videoDetails" (KeyError) Date: 2024-06-02T13:47:51Z Route: /watch?v=27Z4fur1lEg Version: 2024.05.26-bbd550a @ master

Backtrace

``` Missing hash key: "videoDetails" (KeyError) from src/invidious/helpers/helpers.cr:188 in 'dig' from /usr/lib/crystal/core/json/any.cr:342:3 in 'try_fetch_streaming_data' from src/invidious/videos/parser.cr:111:27 in 'extract_video_info' from /usr/lib/crystal/core/hash.cr:1280:13 in 'fetch_video' from src/invidious/videos.cr:383:13 in 'get_video:region' from src/invidious/routes/watch.cr:55:15 in 'handle' from lib/kemal/src/kemal/route.cr:13:9 in '->' from lib/kemal/src/kemal/config.cr:92:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:32:7 in 'call_next' from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:32:7 in 'call_next' from src/invidious/helpers/handlers.cr:94:12 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:32:7 in 'call_next' from /usr/lib/crystal/core/string.cr:3075:20 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:30:7 in 'call' from /usr/lib/crystal/core/http/server/request_processor.cr:51:11 in 'handle_client' from /usr/lib/crystal/core/fiber.cr:146:11 in 'run' from ??? ```

[C-Loftus]

Also getting this same bug for all videos. I am deploying via docker compose on Ubuntu 22.04 for what it is worth. (No configuration on my end)

[ast9501]

I ran into similar problem. Update to newer version works for me: 2024.05.27-1ae14cc2 @ master

[C-Loftus]

I ran into similar problem. Update to newer version works for me: 2024.05.27-1ae14cc2 @ master

I have git cloned the repo on the most recent commit and it says the version is Version: 2023.08.26-ac0c0609 @ master, at least when I build with docker-compose. Could you specify what the right way to get a more recent version would be? Wasn't sure if I can't use docker for that.

[ast9501]

Maybe check your current commit is at latest commit (1ae14cc2)? It seams like you are in older commit on master branch. I built with docker-compose too.

I ran into similar problem. Update to newer version works for me: 2024.05.27-1ae14cc2 @ master

I have git cloned the repo on the most recent commit and it says the version is Version: 2023.08.26-ac0c0609 @ master, at least when I build with docker-compose. Could you specify what the right way to get a more recent version would be? Wasn't sure if I can't use docker for that.

[C-Loftus]

Maybe check your current commit is at latest commit (1ae14cc)? It seams like you are in older commit on master branch. I built with docker-compose too.

I ran into similar problem. Update to newer version works for me: 2024.05.27-1ae14cc2 @ master

I have git cloned the repo on the most recent commit and it says the version is Version: 2023.08.26-ac0c0609 @ master, at least when I build with docker-compose. Could you specify what the right way to get a more recent version would be? Wasn't sure if I can't use docker for that.

Ah that's it, I needed to rebuild the cached version, thanks! docker-compose up -d --no-deps --build.

[Fijxu]

I see no problems on my instance (https://inv.nadeko.net/watch?v=27Z4fur1lEg). Probably youtube is changing the way they display the videoDetails or they are simply blocking some requests made by the instance you are using.

[ZenIsFluffy]

Receiving the same error on my instance as well. I am on 2024.04.10-b673695a @ master and deploying over docker-compose on Debian 12.

Title: Missing hash key: "videoDetails" (KeyError) Date: 2024-06-16T15:59:31Z Route: /watch?v=EnDCumOfgWg Version: 2024.04.10-b673695a @ master

Backtrace

``` Missing hash key: "videoDetails" (KeyError) from /usr/share/crystal/src/hash.cr:1077:11 in '[]' from /usr/share/crystal/src/hash.cr:1128:17 in 'dig' from src/invidious/videos/parser.cr:152:12 in 'try_fetch_streaming_data' from src/invidious/videos/parser.cr:111:5 in 'extract_video_info:video_id' from src/invidious/videos.cr:391:3 in 'fetch_video' from src/invidious/videos.cr:379:13 in 'get_video' from src/invidious/videos.cr:361:1 in 'get_video:region' from src/invidious/routes/watch.cr:55:15 in 'handle' from src/invidious/routing.cr:159:19 in '->' from lib/kemal/src/kemal/route.cr:12:26 in '->' from src/invidious/helpers/handlers.cr:30:37 in 'process_request' from lib/kemal/src/kemal/route_handler.cr:17:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/websocket_handler.cr:13:14 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:157:5 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:94:12 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:146:5 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:70:5 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/ext/kemal_static_file_handler.cr:162:16 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/exception_handler.cr:8:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/logger.cr:17:35 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/init_handler.cr:12:7 in 'call' from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'process' from /usr/share/crystal/src/http/server.cr:517:5 in 'handle_client' from /usr/share/crystal/src/http/server.cr:470:13 in '->' from /usr/share/crystal/src/fiber.cr:146:11 in 'run' from /usr/share/crystal/src/fiber.cr:98:34 in '->' from ??? ```

[Happyfeet01]

Same problem here.

Title: Missing hash key: "videoDetails" (KeyError) Date: 2024-06-22T07:17:07Z Route: /watch?v=F6shQa0yZuE Version: 2024.04.10-b673695 @ master

Backtrace

``` Missing hash key: "videoDetails" (KeyError) from /usr/share/crystal/src/hash.cr:1080:9 in 'dig' from /usr/share/crystal/src/json/any.cr:330:3 in 'try_fetch_streaming_data' from src/invidious/videos/parser.cr:112:10 in 'extract_video_info:video_id' from /usr/share/crystal/src/hash.cr:1185:13 in 'fetch_video' from src/invidious/videos.cr:379:13 in 'get_video:region' from src/invidious/routes/watch.cr:55:15 in 'handle' from lib/kemal/src/kemal/route.cr:13:9 in '->' from src/invidious/helpers/handlers.cr:30:37 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/invidious/helpers/handlers.cr:94:12 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next' from src/ext/kemal_static_file_handler.cr:106:14 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call' from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client' from /usr/share/crystal/src/fiber.cr:146:11 in 'run' from ??? ```

[K33k0]

I'm receiving the same error, using docker compose

docker compose

version: "3"
services:

  invidious:
    image: quay.io/invidious/invidious:latest
    # image: quay.io/invidious/invidious:latest-arm64 # ARM64/AArch64 devices
    restart: unless-stopped
    # ports:
    #   - "127.0.0.1:3000:3000"
    networks:
      - traefik_proxy
    environment:
      # Please read the following file for a comprehensive list of all available
      # configuration options and their associated syntax:
      # https://github.com/iv-org/invidious/blob/master/config/config.example.yml
      INVIDIOUS_CONFIG: |
        db:
          dbname: invidious
          user: kemal
          password: <redacted>
          host: invidious-db
          port: 5432
        check_tables: true
        # external_port:
        # domain: invidious.<redacted>.com
        # https_only: true
        # statistics_enabled: false
        hmac_key: <redacted>
    healthcheck:
      test: wget -nv --tries=1 --spider http://127.0.0.1:3000/api/v1/trending || exit 1
      interval: 30s
      timeout: 5s
      retries: 2
    logging:
      options:
        max-size: "1G"
        max-file: "4"
    labels:
      - "traefik.enable=true"
      # HTTP Routers
      - "traefik.http.routers.invidious-rtr.entrypoints=websecure"
      - "traefik.http.routers.invidious-rtr.rule=Host(`invidious.$DOMAINNAME`)"
      # Middlewares
      - "traefik.http.routers.invidious-rtr.middlewares=chain-authelia@file"
      # HTTP Services
      - "traefik.http.routers.invidious-rtr.service=invidious-svc"
      - "traefik.http.services.invidious-svc.loadbalancer.server.port=3000"

      - "homepage.group=Media"
      - "homepage.name=Invidious"
      - "homepage.icon=invidious.png"
      - "homepage.href=https://invidious.<redacted>.com"

    depends_on:
      - invidious-db

  invidious-db:
    image: docker.io/library/postgres:14
    restart: unless-stopped
    volumes:
      - $DATADIR/invidious/postgresdata:/var/lib/postgresql/data
      - $DATADIR/invidious/sql:/config/sql
      - $DATADIR/invidious/docker/init-invidious-db.sh:/docker-entrypoint-initdb.d/init-invidious-db.sh
    environment:
      POSTGRES_DB: invidious
      POSTGRES_USER: kemal
      POSTGRES_PASSWORD: <redacted>
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
    networks:
      - traefik_proxy

Error

Title: `Missing hash key: "videoDetails" (KeyError)`
Date: `2024-06-24T11:52:54Z`
Route: `/watch?v=0psrAMafXVg`
Version: `2024.03.31-08390ac @ master`

<details>
<summary>Backtrace</summary>
<p>

\```
Missing hash key: "videoDetails" (KeyError)
  from /usr/share/crystal/src/hash.cr:1080:9 in 'dig'
  from /usr/share/crystal/src/json/any.cr:330:3 in 'try_fetch_streaming_data'
  from src/invidious/videos/parser.cr:112:10 in 'extract_video_info:video_id'
  from /usr/share/crystal/src/hash.cr:1185:13 in 'fetch_video'
  from src/invidious/videos.cr:379:13 in 'get_video:region'
  from src/invidious/routes/watch.cr:55:15 in 'handle'
  from lib/kemal/src/kemal/route.cr:13:9 in '->'
  from src/invidious/helpers/handlers.cr:30:37 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/invidious/helpers/handlers.cr:94:12 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call_next'
  from src/ext/kemal_static_file_handler.cr:106:14 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/handler.cr:30:7 in 'call'
  from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'handle_client'
  from /usr/share/crystal/src/fiber.cr:146:11 in 'run'
  from ???
\```
</p>
</details>

[unixfox]

This is because sometimes the client ANDROID_TESTSUITE does not work anymore.

It seems like youtube is A/B testing the removal of this client.

[K33k0]

Fixed for me with the latest docker image quay.io/invidious/invidious:latest image (07ceda5d18d4) Current version: 2024.04.27-eda7444 @ master

[ZenIsFluffy]

I feel kinda silly. I had been running docker-compose pull but didn't think to run git pull. Anyway, after that I ran docker-compose up -d --no-deps --build as mentioned above. Brought my version from 2024.04.10-b673695a @ master to 2024.05.27-1ae14cc2 @ master and now its working!

[Happyfeet01]

I feel kinda silly. I had been running docker-compose pull but didn't think to run git pull. Anyway, after that I ran docker-compose up -d --no-deps --build as mentioned above. Brought my version from 2024.04.10-b673695a @ master to 2024.05.27-1ae14cc2 @ master and now its working!

I do the same but still on 2024.04.10-b673695 @ master

[Fijxu]

I feel kinda silly. I had been running docker-compose pull but didn't think to run git pull. Anyway, after that I ran docker-compose up -d --no-deps --build as mentioned above. Brought my version from 2024.04.10-b673695a @ master to 2024.05.27-1ae14cc2 @ master and now its working!

I do the same but still on 2024.04.10-b673695 @ master

Do git pull first, then build the image.

[unixfox]

Don't use the docker-compose.yml from the repo please. Use the production docker-compose: https://docs.invidious.io/installation/

[Happyfeet01]

I forgot the docker-compose pull after the git pull. Now it works ;-)