iv-org / invidious

Invidious is an alternative front-end to YouTube
https://invidious.io
GNU Affero General Public License v3.0
16.42k stars 1.85k forks source link

[Bug] Expected Hash for #[]?(key : String), not Array(JSON::Any) (Exception) #3352

Closed AHOHNMYC closed 1 year ago

AHOHNMYC commented 2 years ago

From time to time I got this error. Especially at first try to load a video. Instance: yewtu.be Sorry if this is a double.

Title: Expected Hash for #[]?(key : String), not Array(JSON::Any) (Exception) Date: 2022-10-19T13:26:16Z Route: /watch?v=8Ed85yna_FQ Version: 2022.10.19-2dfe9d9 @ master

Backtrace

``` Expected Hash for #[]?(key : String), not Array(JSON::Any) (Exception) from src/invidious/yt_backend/extractors.cr:483 in '[]?' from /usr/lib/crystal/core/json/any.cr:124:5 in 'parse_video_info' from src/invidious/videos/parser.cr:92:12 in 'extract_video_info:video_id' from /usr/lib/crystal/core/hash.cr:1185:13 in 'fetch_video' from src/invidious/videos.cr:346:17 in 'get_video:region' from src/invidious/routes/watch.cr:63:15 in 'handle' from lib/kemal/src/kemal/route.cr:13:9 in '->' from lib/kemal/src/kemal/config.cr:92:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call' from /usr/lib/crystal/core/http/server/handler.cr:28:7 in 'call_next' from lib/kemal/src/kemal/init_handler.cr:12:7 in 'process' from /usr/lib/crystal/core/http/server.cr:500:5 in '->' from /usr/lib/crystal/core/fiber.cr:146:11 in 'run' from /src/boringssl/src/google-boringssl-251b516/ssl/internal.h:334:8 in '??' ```

unixfox commented 2 years ago

Probably due to testing of PR https://github.com/iv-org/invidious/pull/3238 I've removed the testing of this PR for the moment.

unixfox commented 2 years ago

Caused by https://github.com/iv-org/invidious/pull/3238, closing for now.

SamantazFox commented 2 years ago

Weird, I can't reproduce with the given video and the mentioned PR deployed.

@unixfox this is probably because you're running concurrently two cache schemes in your DB?

SamantazFox commented 2 years ago

I think I got a better trace:

Title: Expected Hash for #[]?(key : String), not Array(JSON::Any) (Exception) Date: 2022-10-24T22:50:18Z Route: /watch?v=qZ2GRTiCdeA Version: 2022.09.28-6707368f @ master

Backtrace

``` Expected Hash for #[]?(key : String), not Array(JSON::Any) (Exception) from /usr/share/crystal/src/json/any.cr:115:7 in '[]?' from /usr/share/crystal/src/json/any.cr:124:5 in 'dig?' from invidious.git/src/invidious/videos/parser.cr:239:45 in 'parse_video_info' from invidious.git/src/invidious/videos/parser.cr:92:3 in 'extract_video_info:video_id' from invidious.git/src/invidious/videos.cr:366:3 in 'fetch_video' from invidious.git/src/invidious/videos.cr:354:13 in 'get_video' from invidious.git/src/invidious/videos.cr:336:1 in 'get_video:region' from invidious.git/src/invidious/routes/watch.cr:63:15 in 'handle' from invidious.git/src/invidious/routing.cr:139:19 in '->' from invidious.git/lib/kemal/src/kemal/route.cr:12:26 in '->' from invidious.git/src/invidious/helpers/handlers.cr:30:5 in 'process_request' from invidious.git/lib/kemal/src/kemal/route_handler.cr:17:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/lib/kemal/src/kemal/websocket_handler.cr:13:14 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/lib/kemal/src/kemal/filter_handler.cr:21:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/invidious/helpers/handlers.cr:212:5 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/invidious/helpers/handlers.cr:94:12 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/invidious/helpers/handlers.cr:145:12 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/invidious/helpers/handlers.cr:70:5 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/ext/kemal_static_file_handler.cr:162:16 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/lib/kemal/src/kemal/exception_handler.cr:8:7 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/src/invidious/helpers/logger.cr:17:35 in 'call' from /usr/share/crystal/src/http/server/handler.cr:28:7 in 'call_next' from invidious.git/lib/kemal/src/kemal/init_handler.cr:12:7 in 'call' from /usr/share/crystal/src/http/server/request_processor.cr:51:11 in 'process' from /usr/share/crystal/src/http/server.cr:515:5 in 'handle_client' from /usr/share/crystal/src/http/server.cr:468:13 in '->' from /usr/share/crystal/src/fiber.cr:146:11 in 'run' from /usr/share/crystal/src/fiber.cr:98:34 in '->' from ??? ```

SamantazFox commented 2 years ago

Reopening, as Youtube is apparently A/B testing something (and I managed to catch it).

Response of the next endpoint (before)

{
  "contents": {
    "twoColumnWatchNextResults": {
      "results": {
        "results": {
          "contents": [
            {
              "videoPrimaryInfoRenderer": {
                "title": {
                  "runs": [
                    {
                      "text": "Aspargus II - A 16-bit Minecraft redstone computer fully programmable with assembler"
                    }
                  ]
                },
                "viewCount": {
                  "videoViewCountRenderer": {
                    "viewCount": {
                      "simpleText": "1,332 views"
                    },
                    "shortViewCount": {
                      "simpleText": "1.3K views"
                    }
                  }
                },
                "videoActions": {
                  "menuRenderer": {
                    "items": [],
                    "topLevelButtons": [
                      {
                        "toggleButtonRenderer": {
                          "style": {
                            "styleType": "STYLE_TEXT"
                          },
                          "isToggled": false,
                          "isDisabled": false,
                          "defaultIcon": {
                            "iconType": "LIKE"
                          },
                          "defaultText": {
                            "accessibility": {
                              "accessibilityData": {
                                "label": "48 likes"
                              }
                            },
                            "simpleText": "48"
                          },

And now:

{
  "contents": {
    "singleColumnWatchNextResults": {
      "results": {
        "results": {
          "contents": [
            {
              "slimVideoMetadataSectionRenderer": {
                "contents": [
                  {
                    "slimVideoInformationRenderer": {
                      "title": {
                        "runs": [
                          {
                            "text": "Aspargus II - A 16-bit Minecraft redstone computer fully programmable with assembler"
                          }
                        ]
                      },
                      "collapsedSubtitle": {
                        "runs": [
                          {
                            "text": "1.3K views"
                          }
                        ],
                        "accessibility": {
                          "accessibilityData": {
                            "label": "1,332 views"
                          }
                        }
                      },
                      "expandedSubtitle": {
                        "runs": [
                          {
                            "text": "1,332 views"
                          }
                        ]
                      },
                      "caretStyle": "SLIM_VIDEO_INFORMATION_CARET_STYLE_ALIGN_TO_CORNER"
                    }
                  },
                  {
                    "slimVideoScrollableActionBarRenderer": {
                      "buttons": [
                        {
                          "slimMetadataToggleButtonRenderer": {
                            "target": {
                              "videoId": "qZ2GRTiCdeA"
                            },
                            "isLike": true,
                            "button": {
                              "toggleButtonRenderer": {
                                "isToggled": false,
                                "isDisabled": false,
                                "defaultIcon": {
                                  "iconType": "LIKE"
                                },
                                "defaultText": {
                                  "runs": [
                                    {
                                      "text": "48"
                                    }
                                  ],
                                  "accessibility": {
                                    "accessibilityData": {
                                      "label": "48 likes"
                                    }
                                  }
                                },
SamantazFox commented 2 years ago

So I've tracked down the issue, it's not related at all to my last message.

It was the like button renderer:

{
  "segmentedLikeDislikeButtonRenderer": {
    "likeButton": {
      "toggleButtonRenderer": {
        ...
      }
    }
  }
},
SamantazFox commented 1 year ago

This was fixed in the aforementioned PR.