XRPCError: fetch failed

[grasshoppermouse]

/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/types.js:174
        return new XRPCError(status, undefined, message, headers, { cause });
               ^

XRPCError: fetch failed
    at XRPCError.from (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/types.js:174:16)
    at AtpAgent.call (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/xrpc-client.js:89:37)
    at async RateLimitedAgent.call (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/app.js:90:24)
    at async main (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/app.js:511:52) {
  error: 'Unknown',
  headers: undefined,
  success: false,
  status: 1,
  [cause]: TypeError: fetch failed
      at node:internal/deps/undici/undici:13178:13
      at async CredentialSession.fetchHandler (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/@atproto/api/dist/atp-agent.js:188:28)
      at async AtpAgent.call (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/@atproto/xrpc/dist/xrpc-client.js:67:30)
      at async RateLimitedAgent.call (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/app.js:90:24)
      at async main (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/app.js:511:52) {
    [cause]: SocketError: other side closed
        at TLSSocket.<anonymous> (/Users/hagen/Documents/Misc/twitterstuff/twitter-to-bluesky/node_modules/undici/lib/dispatcher/client-h1.js:701:24)
        at TLSSocket.emit (node:events:531:35)
        at endReadableNT (node:internal/streams/readable:1696:12)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      code: 'UND_ERR_SOCKET',
      socket: {
        localAddress: '192.168.30.178',
        localPort: 62066,
        remoteAddress: '15.204.101.248',
        remotePort: 443,
        remoteFamily: 'IPv4',
        timeout: undefined,
        bytesWritten: 444925,
        bytesRead: 1807
      }
    }
  }
}

Node.js v20.16.0

Last item in log:

Parse tweet id '1692286776206057667'
 Created at 2023-08-17T21:26:23.000Z
 Full text '@HamesRaymond @huw_groucutt The question is, are those distributions significantly different, not from 0, but from each other? I just downloaded the data and tested for an interaction (using frequentist, not Bayes) & no, they're not, p = 0.17 (caveat: quick & dirty--needs more careful analysis to confirm): https://t.co/VVJzL46VEw'
Saving already imported tweets to tweets_mapping.json

[clemensv]

Thank you for reporting that. This should be healed by restarting; does this happen more often? If so, we could put a retry into RateLimitedAgent.

[grasshoppermouse]

I got this error two more times, so a total of three times for ~1000 imported tweets.

As an aside, I got a lot of these (about 3x as many as in this screenshot):

[clemensv]

The 403s are all expired links.

[clemensv]

I am trying this modification the rate limited agent. I've seen this once, so I am hoping I see it again.

async call<T>(method: () => Promise<T>): Promise<T> {
        let attempts = 0;
        while (true) {
            try {
                if (this.waitingForRateLimit) {
                    await new Promise(resolve => setTimeout(resolve, 1000));
                    continue;
                }
                return await method();
            } catch (error: any) {
                if ( ++attempts > 5) {
                    throw error;
                }
                if (error.message.includes('fetch failed')) {
                    console.warn(`Fetch failed, retrying attempt ${attempts}/5...`);
                    await new Promise(resolve => setTimeout(resolve, 2000));
                    continue;
                }
                if (error.status === 429) {
                    await this.handleRateLimit(error);
                } else {
                    throw error;
                }
            }
        }
    }

[clemensv]

Lucky!

Parse tweet id '760390048588136448'
 Created at 2016-08-02T08:21:29.000Z
 Full text 'Bonjour Asterix! (@ Parc Astérix in Plailly, Picardie) https://t.co/YHSeB53C1U https://t.co/OA6xbLkdWE'
**Fetch failed, retrying attempt 1/5...**
 Clean text 'Bonjour Asterix! (@ Parc Astérix in Plailly, Picardie) https://www.swarmapp.com/c/fLeSRS7g2Lz'
Error fetching oembed: no oembed discovery information available

[clemensv]

@marcomaroni-github I believe this is handled

[marcomaroni-github]

@marcomaroni-github I believe this is handled

@clemensv so can I close the bug?