search

snarfed / bridgy-fed

🌉 A bridge between decentralized social network protocols
https://fed.brid.gy
Creative Commons Zero v1.0 Universal
553 stars 29 forks source link

upgrade libipld dep to 2.0.0 #1316

Open snarfed opened 1 week ago

snarfed commented 1 week ago

It's faster! ...but has a breaking change, it returns blob CIDs as raw bytes, not base-32 encoded strings. https://github.com/MarshalX/python-libipld/releases/tag/v2.0.0

It was straightforward to add support for this to granary.bluesky.blob_to_url in snarfed/granary@59b207cd8b8d7c6c2f2975601ff8427650f1f516, but it's not as easy to support it in the Object.bsky datastore property. It's a webutil.models.JsonProperty, which JSON-encodes its value. That class doesn't know anything about ATProto or CIDs, and I don't plan to make it assume that bytes values are CIDs...but the other option is to walk all objects somewhere in BF, eg in atproto_firehose._handle, find all of the blob fields (by checking against lexicons?), and convert their bytes to CIDs. Basically a libipld post-processing to apply to some records to convert CIDs to multiformats.CID instances. Seems like overkill. Hrmph.

snarfed commented 1 week ago

For when I do come back to this, these are the errors we saw in prod: