Integrate Granary as Middleware for AT Protocol Interoperability

outlaw-dame commented 3 months ago

Summary

This proposal outlines a plan to integrate Granary as middleware within ActivityPods to enable interoperability with the AT Protocol. The goal is to bridge ActivityPods, which utilises ActivityPub and Solid standards, with the emerging AT Protocol, thereby enhancing cross-platform communication and user interaction across decentralised social networks. https://github.com/snarfed/granary

Background

Granary is a library that converts data between different social web formats, including ActivityStreams, microformats, HTML, and JSON. Extending Granary to support AT Protocol (https://github.com/bluesky-social/atproto)and integrating it as middleware in ActivityPods can achieve seamless interoperability between ActivityPods-based applications and AT Protocol platforms.

Objectives

Enhance Interoperability: Enable ActivityPods applications to communicate with AT Protocol platforms, breaking down silos in the decentralised social web.
User Empowerment: Allow users to interact across different platforms without duplicating their social graph or content.
Developer Support: Provide developers with tools to build applications that can operate across different decentralised social networking protocols.

Proposed Approach

Phase 1: Planning and Analysis

Understand the protocols (ActivityPub, Solid, AT Protocol) and identify key integration points.
Map common entities and actions between ActivityStreams and AT Protocol lexicons.

Phase 2: Extension of Granary

Develop AT Protocol support in Granary, focusing on conversion logic for core entities like users and posts.
Handle extensions and custom fields specific to ActivityStreams and AT Protocol.

Phase 3: Middleware Integration

Design and implement middleware architecture within ActivityPods to utilise Granary for data conversion.
Adapt ActivityPods' API endpoints for AT Protocol communication, focusing on authentication and data flow.

Phase 4: Testing and Validation

Conduct comprehensive testing, including unit, functional, and interoperability tests.
Set up a test environment for real-world scenario testing.

Phase 5: Documentation and Deployment

Document the integration process, API usage, and contribution guidelines.
Develop a deployment strategy and monitor the integration's performance.

Request for Comments

I invite the ActivityPods community to discuss this proposal. Feedback on the approach, potential challenges, and additional benefits are welcome. Collaboration is key to achieving interoperability and enhancing the decentralised social web.

Conclusion

Integrating Granary as middleware to achieve interoperability with AT Protocol represents a significant step towards a more interconnected and user-centric decentralised social web. This proposal aims to start a collaborative effort towards this goal, leveraging the strengths of ActivityPods, Granary, and the broader open-source community.

Thank you for considering this proposal. I look forward to your feedback and the opportunity to work together on this exciting integration.

I understand this a large undertaking. I in no way believe this to be addressed nor completed anytime soon but the possibilities are incredible. ActivityPods would garner rapid adoption

outlaw-dame commented 2 months ago

Adding this to provide context for Granary and what it covers. Granary is a library and REST API that fetches and converts between a wide variety of social data sources and formats:

Facebook, Flickr, GitHub, Instagram, Mastodon, and Twitter native APIs Instagram and Facebook scraped HTML ActivityStreams 1.0 and 2.0 JSON, including ActivityPub HTML and JSON with microformats2 Atom, RSS 2.0, JSON Feed Plain XML Bluesky/AT Protocol Nostr, with many NIPs Free yourself from silo API chaff and expose the sweet social data foodstuff inside in standard formats and protocols!

snarfed commented 2 months ago

Thanks for the idea and vote of support @outlaw-dame!

Granary can indeed convert social data to Bluesky format, ie the app.bsky.* lexicons, which can then be served over AT Protocol. Actually supporting AT Protocol itself, ie acting as a full fledged PDS, is a bigger lift: you need to store and serve MST nodes and a repo commit chain over websocket. You can do that with eg https://github.com/snarfed/arroba , which integrates well with granary, or you can write your data to a standalone PDS, either one you run (the official one is open source) or someone else's like the official https://bsky.social/ .

outlaw-dame commented 2 months ago

Thanks for the idea and vote of support @outlaw-dame!

Granary can indeed convert social data to Bluesky format, ie the app.bsky.* lexicons, which can then be served over AT Protocol. Actually supporting AT Protocol itself, ie acting as a full fledged PDS, is a bigger lift: you need to store and serve MST nodes and a repo commit chain over websocket. You can do that with eg https://github.com/snarfed/arroba , which integrates well with granary, or you can write your data to a standalone PDS, either one you run (the official one is open source) or someone else's like the official https://bsky.social/ .

How would this be used to map the data/PDS to the Pod Store which is the ActivityPods backend and where user data is housed

snarfed commented 2 months ago

You'd integrate arroba into the ActivityPods backend to create ATProto users, repos and create, update, and delete records in those repos when you do the corresponding things for native ActivityPods users. You could optionally also write a new implementation of arroba's Storage class that stores repo data in the ActivityPods data store.

However, it looks like ActivityPods is JavaScript, and arroba is Python, so you'd have to use cross-language bindings, which can sometimes be awkward. The official PDS is JavaScript, so you might want to look at using that instead.

assemblee-virtuelle / activitypods