posts are very delayed

[wbz0100]

Its delayed so much. Is this normal? I think it's been lagging a lot since the last time it was disconnected.

Instead of being posted sequentially. And In severe cases, there is a delay of up to 10 hours.

[snarfed]

Yes! Not normal, we're badly backed up right now. Started around midnight PT last night, our send task queue peaked at over 3M tasks (!). Sorry for the trouble. It's down to 2.5M ish now, but still. 😕

I'm not sure what's going on yet. Coincidentally, I got pretty sick last night, and I'm still out of it, so I haven't been able to look yet. Hopefully soon.

[snarfed]

For my own notes, here's the repl code I'm using to clear out the send queue:

from google.cloud import tasks_v2
tasks_client = tasks_v2.CloudTasksClient()

from common import TASKS_LOCATION
from google.cloud.tasks_v2.types import Task

request = tasks_v2.ListTasksRequest(
    parent=tasks_client.queue_path(appengine_info.APP_ID, TASKS_LOCATION, 'send'),
    response_view=Task.View.FULL,
)

deleted = 0
for resp in tasks_client.list_tasks(request=request):
  if urllib.parse.parse_qs(resp.app_engine_http_request.body).get(b'user') == [b'ahBicmlkZ3ktZmVkZXJhdGVkci0LEgdBVFByb3RvIiBkaWQ6cGxjOm92eWFqYm5lZ3JpcjJkZHR3ZmhpamlvYgw']:
    deleted += 1
    print(deleted)
    tasks_client.delete_task(request=tasks_v2.DeleteTaskRequest(name=resp.name))

[snarfed]

We're almost entirely caught up, send task queue is under 10k now. Thanks for your patience all!

[imsodin]

Thanks for dealing with this so quickly!

For my own notes, here's the repl code I'm using to clear out the send queue:

Does that mean some posts during the incident were not replicated?
I am just asking to get an idea if some problems I just heared about might be related or not. I had a chat with someone who unfortunately started trying out the bridge more or less at the same time as these troubles happened.

[snarfed]

Hopefully we didn't drop any posts. Let me know if you see any that are missing and pass the checks in https://fed.brid.gy/docs#troubleshooting , happy to look.

[fadookie]

@snarfed thanks for handling this. Is it possible this is happening again? In my fedi -> bsky bridged profile I haven't seen any posts updating in over 9 hours (I'm not sure what the usual delay is): https://bsky.app/profile/Eliot-L.social.coop.ap.brid.gy

[wbz0100]

Umm... Looks like it's happening now. A post from about 2 hours ago is publishing now.

This seems to happen constantly, and I'm not sure if it's because the post speed can't keep up with the processing speed, or if there's something wrong in the processing.

[EmilJacobs]

Can confirm, this issue isn't resolved.

[avocado1969]

I’m also not seeing fedi actions populate onto bsky since yesterday evening, even though my profile on fed.brid.gy shows the changes going though with the green check

[dakota-marshall]

Not to pile on, but I can confirm this is still happening aswell. I have severl public posts and replies that all seem to have the green check, but are not appearing on the Bluesky account.

I actually haven't had anything sync over since I bridged my account like 24 hours ago

[Ralph1798]

I followed a newly bridged account and posts from 350d+ started to appear in my Mastodon-timeline. Interesting enough: no current posts, just the old ones.

[snarfed]

Not to pile on, but I can confirm this is still happening aswell. I have severl public posts and replies that all seem to have the green check, but are not appearing on the Bluesky account.

I actually haven't had anything sync over since I bridged my account like 24 hours ago

Yes! Sorry, reopening. The current Bluesky => fediverse delay is on us, fediverse => Bluesky delay is on Bluesky team. We're (all) working on it. 😬

[wbz0100]

I know you've been sick, so please take it easy and work slowly.

I think one of the causes may have been a delay in moving posts from X (formerly Twitter) to Bluesky.

[dakota-marshall]

Not to pile on, but I can confirm this is still happening aswell. I have severl public posts and replies that all seem to have the green check, but are not appearing on the Bluesky account. I actually haven't had anything sync over since I bridged my account like 24 hours ago

Yes! Sorry, reopening. The current Bluesky => fediverse delay is on us, fediverse => Bluesky delay is on Bluesky team. We're (all) working on it. 😬

Thanks for the response! No one is dying, so this is such a non-issue in the grand scheme of things. Hope you feel better :) Thanks for such a cool tool :D

[Daft-Freak]

How much delay should I be expecting on the bluesky side? (I haven't seen a post I've made get through in the last 24 hours and am wondering if I have an actual problem...)

[snarfed]

@Daft-Freak you mean fediverse => Bluesky? They're doing a lot of work on their relay, https://github.com/bluesky-social/atproto/discussions/3036 , and they've temporarily paused ingesting commits from federated PDSes like Bridgy Fed. I don't have an ETA for when that will be lifted, sadly.

[Daft-Freak]

Ah okay, good to know!

[snarfed]

https://bsky.app/profile/bnewbold.net/post/3lbimsarmw22g https://bsky.app/profile/bnewbold.net/post/3lbipqvij7k2m https://bsky.app/profile/bnewbold.net/post/3lbirrqme2s2w

[wbz0100]

Maybe bsky to fedi has been normalized. I think, but I don't think the reverse is yet, but I think they've blocked it.