XRPCError: This file is too large

[iznaut]

/Users/izzy/Repos/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/client.js:106
                throw new types_1.XRPCError(resCode, res.body.error, res.body.message, res.headers);
                      ^

XRPCError: This file is too large. It is 1017.52KB but the maximum size is 976.56KB.
    at ServiceClient.call (/Users/izzy/Repos/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/client.js:106:23)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async PostRecord.create (/Users/izzy/Repos/twitter-to-bluesky/node_modules/@atproto/api/dist/client/index.js:1524:21)
    at async main (/Users/izzy/Repos/twitter-to-bluesky/app.js:216:36) {
  status: 400,
  error: 'BlobTooLarge',
  success: false,
  headers: {
    'access-control-allow-origin': '*',
    'cache-control': 'private',
    'content-length': '110',
    'content-type': 'application/json; charset=utf-8',
    date: 'Tue, 22 Oct 2024 18:50:34 GMT',
    etag: 'W/"6e-70AZiBcSZ2n1SgjKj7kIhAwzpEc"',
    'keep-alive': 'timeout=90',
    'ratelimit-limit': '5000',
    'ratelimit-policy': '5000;w=3600',
    'ratelimit-remaining': '4931',
    'ratelimit-reset': '1729626558',
    'strict-transport-security': 'max-age=63072000',
    vary: 'Authorization, Accept-Encoding',
    'x-powered-by': 'Express'
  }
}

Node.js v18.20.3

[marcomaroni-github]

@iznaut Unfortunately 1 MB is the limit currently imposed by the API see https://docs.bsky.app/docs/advanced-guides/posts#images-embeds and this discussion https://github.com/bluesky-social/atproto/discussions/1740

[iznaut]

Sorry, I should have added more context - I understand that it's a Bluesky limitation, but it would be nice if the error could be handled more gracefully?

I'd be fine if it just skipped over these tweets, but right now it ends the process completely so it's impossible to finish my import.

[marcomaroni-github]

Sorry, I should have added more context - I understand that it's a Bluesky limitation, but it would be nice if the error could be handled more gracefully?

I'd be fine if it just skipped over these tweets, but right now it ends the process completely so it's impossible to finish my import.

Ok now it's clear, until the library is stable I preferred this approach so that it would stop and I could analyze the errors in detail. While I evaluate how to handle these cases, you can continue the import using the MIN_DATE parameter, setting it to a date a few seconds more recent than the tweet you want to skip.

[iznaut]

Ok now it's clear, until the library is stable I preferred this approach so that it would stop and I could analyze the errors in detail. While I evaluate how to handle these cases, you can continue the import using the MIN_DATE parameter, setting it to a date a few seconds more recent than the tweet you want to skip.

I actually tried just that when it first failed, haha. It got a few more tweets in before crashing on another big image and I decided to give up for the moment.

Thinking about it again, though - since this is using the local Twitter archive on my machine, I could probably just try to compress all the large images before running it again? 🤔

[readtedium]

Another idea I might suggest is putting a timeout on tweets with delays, so if it’s holding up the script it doesn’t just put it in stasis. Two minutes seems like a reasonable timeout period.