Certain remote media files fail to load.

[pilar6195]

💡 Summary

I recently updated my instance to the most recent version (v13.11.3) and right after I noticed that some user's profile avatars were not loading. Checking the network tab would show a http 500 error for the image.

I also noticed that new posts containing images from these same users are no longer appearing on my instance. I confirmed that there indeed were new posts by viewing the user's profile on the remote instance directly. The posts that did not appear on my instance all contained images.

I believe that this is related to how misskey handles remote media as this seems to only affect certain remote media urls.

This also happens on the main misskey.io instance.

🥰 Expected Behavior

The user's avatar image and posts containing media should load.

🤬 Actual Behavior

The user's avatar image throws a http 500 error and any posts containing media does not appear.

Here is the error that I get in the server console whenever the affected remote profile avatar/post containing media is fetched.

RequestError: invalid type format
      at Request._beforeError (file:///misskey/node_modules/.pnpm/got@12.6.0/node_modules/got/dist/source/core/index.js:310:21)
      at Request._onResponse (file:///misskey/node_modules/.pnpm/got@12.6.0/node_modules/got/dist/source/core/index.js:769:18)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at parse (/misskey/node_modules/.pnpm/content-disposition@0.5.4/node_modules/content-disposition/index.js:322:11)
      at Request.<anonymous> (file:///misskey/packages/backend/built/core/DownloadService.js:91:32)
      at Request.emit (node:events:513:28)
      at Request._onResponseBase (file:///misskey/node_modules/.pnpm/got@12.6.0/node_modules/got/dist/source/core/index.js:730:14)
      at Request._onResponse (file:///misskey/node_modules/.pnpm/got@12.6.0/node_modules/got/dist/source/core/index.js:765:24)
      at ClientRequest.<anonymous> (file:///misskey/node_modules/.pnpm/got@12.6.0/node_modules/got/dist/source/core/index.js:783:23)
      at Object.onceWrapper (node:events:628:26)
      at ClientRequest.emit (node:events:525:35)
      at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:695:27)
      at HTTPParser.parserOnHeadersComplete (node:_http_common:117:17)
      at TLSSocket.socketOnData (node:_http_client:536:22)
      at TLSSocket.emit (node:events:513:28)
      at addChunk (node:internal/streams/readable:324:12)
      at readableAddChunk (node:internal/streams/readable:297:9)
      at Readable.push (node:internal/streams/readable:234:10)
      at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

I was able to track down the error to the following line.

https://github.com/misskey-dev/misskey/blob/c5d2dba28d1e99f9c152840ca7f47e8f51c7423b/packages/backend/src/core/DownloadService.ts#L89

Based on this I believe the issue is a result of this commit 2d551a8598de12210ddb7f708561e51867ce3f10. If I revert to v13.9.1 (version right before this commit) the issue disappears and affected remote user avatars and media posts load again.

One of the accounts that is affected, and I tested with is @NekoBot@bae.st

Should also note that I tested this on a fresh instance on both the latest and v13.9.1 versions.

📝 Steps to Reproduce

1. Follow the affected account mentioned above. Also notice that the account's avatar does not load.
2. Wait for a media post that happens every 30 minutes. Can confirm by checking remote instance.
3. Notice that the post doesn't appear on your/misskey.io instance and the error that appears in the server console when it attempted to fetch the post's image.

📌 Environment

Misskey version: 13.11.2 Your OS: Windows 10 22H2 (19045.2846) Your browser: Microsoft Edge 114.0.1788.0 (Official build) dev (64-bit)

[pilar6195]

Looked into it a bit more and the issue occurs is if the type is missing from the Content-Disposition header.

All of the media files that I noticed this error occurring on have the Content-Disposition header formatted like filename="some_filename.png" instead of something like attachment; filename="some_filename.png". If the type is missing the parse function throws that invalid type format error.

[900809]

i have this issue also in recent versions

[tamaina]

https://bae.st/media/de5cce5aca9084b2ba78a24f5acd655934115987d74699e36c8850fa9038fe02.png?name=chocola_nieveexcitement.png

なるほど

[saschanaz]

つまりMastodonのバグですか？

Edit: Pleromaか https://git.pleroma.social/pleroma/pleroma/-/blob/e853cfe7c3438650fd0f95bfe69e2bccfe12390c/lib/pleroma/web/plugs/uploaded_media.ex#L40

[tamaina]

規格外だからまあPleromaのバグ？

（HTTPヘッダーなんて規格外の値が突っ込まれるのは割とあるのでこのくらいエラーにしないでおくれと思った）

[tamaina]

他にも原因がある気がするけどとりあえずinvalidとしてマークしてclose

[saschanaz]

Filed https://git.pleroma.social/pleroma/pleroma/-/issues/3114

[pilar6195]

Yeah, I realized after that it's not exactly an issue with misskey but regardless there should still be better error handling for these cases which I see has been done now. 👍