Plugin: Send blog entries to a PubSub nodes

sjehuda commented 2 months ago

Greetings!

In XMPP, using PubSub(1) (XEP-0060), it is possible to store posts as Atom Syndication(2) entries into XMPP(3).

There are several XMPP extensions that apply it, namely XEP-0227 (4) and XEP-0472 (5) which make node urn:xmpp:microblog:0 as a common choice to manage news for client accounts (e.g. blog@chyrplite.net), yet a component account (e.g. pubsub.chyrplite.net) may use all of its nodes for this purpose.

pubsub.chyrplite.net/
|
|-- updates/
    |-- Sombre
     `-- Varied
|-- releases/
 `-- events/

pubsub service/
|
|-- node/
    |-- item
     `-- item
|-- node/
 `-- node/

A PHP library that can be utilized would be Moxl of Movim.

I have made a script that realizes viewing PubSub node items as Atom XML feeds over HTTP.

Neustradamus commented 2 months ago

To follow this ticket :)

xenocrat commented 2 months ago

Hello there,

Thank you for the post and all the links. To be honest, I'm not sure that I understand all of this! Can you explain in detail what your suggested enhancement is, with an example use case?

sjehuda commented 2 months ago

@xenocrat thank you for giving me the opportunity to explain myself.

TL;DR; I suggest to use XMPP (and Atom) as an additional hosting storage to HTTP (and HTML).

I am not sure I know how to make it more detailed without losing the observer's "train of thought".

Instead, I will make it as concise as possible:

Introduction

There is a messaging and presence standard protocol called XMPP (RFC 6121).
This protocol - which is entirely structured from XML - has extensions called (XEPs).
There is an XEP known as "PubSub" which allows to upload contents to XMPP.

Idea

PubSub allows various of use cases, and one of which is uploading Atom Syndication feeds which are also XML.
I suggest to make use of XMPP as an additional mean to publish posts and websites.

Realization

The Movim platform implements the above idea and has ported tens of news sources to PubSub nodes.
The posts of websites that provide Atom feeds look exactly or almost exactly as the originals.
All data is stored as Atom feeds inside XMPP PubSub nodes (as I noted, both are XML).

Possible use case

There are many.
One that I can think of is this:
Two friends - Amanda and Kelly - are chatting via XMPP.
While chatting Amanda checks the PubSub node associated with the one of her friend Kelly to see what news Kelly has published recently.
Alternatively, PubSub also supports push notifications, so Amanda can be notified or get an indication by Kelly's PubSub about new posts.
In this use-case, PubSub can improve rapport between friends.

Additional context

Mr. @roughnecks - who is a great sysadmin I admire - runs Chyrp Lite at woodpeckersnest.space.
He is also hosting XMPP and Movim instances with which we can test that feature, if you would want to.

xenocrat commented 2 months ago

Thanks for the details. So, if I understand correctly I need to add three things to Chyrp Lite in order to support Atom syndication over PubSub:

A way to generate an Atom entry when a blog post is created/updated;
Moxl or another library that will allow me to publish the blog entry to a chosen XMPP node;
A settings page for the blog administrator to enter XMPP node URL and account credentials to authorize publishing.

Numbers 1 and 3 are easy enough. Number 2 depends on how onerous the library is to use. I see Google have a PHP PubSub implementation but its requirements are a no-go for this project. At first glance, Moxl seems to have lighter requirements, but I need to check how far the dependency chain goes. Leave this with me - I'll report back on this thread.

sjehuda commented 2 months ago

A way to generate an Atom entry when a blog post is created/updated;

This is a good idea.

Moxl or another library that will allow me to publish the blog entry to a chosen XMPP node;

This is correct.

A settings page for the blog administrator to enter XMPP node URL and account credentials to authorize publishing.

Yes, with a note that says "be sure the (XMPP) account has the necessary permissions to publish into the chosen pubsub service"

Moxl seems to have lighter requirements, but I need to check how far the dependency chain goes. Leave this with me - I'll report back on this thread.

Note concerning to 1

In case of constrains:

Since each pubsub node item has its own ID, it is possible to upload the whole Atom feed. Each conflict would result in replacing each current item with a current uploaded item.

Note concerning to 2

1) The important phase of listening and understanding the matter appears to be over. 2) The lesser important - albeit still of high importance - phase is applying XMPP support.

We can wait a while for second phase. Please, take your time.

sjehuda commented 2 months ago

I suggest you to ask "edhelas" from Movim about Moxl.

sjehuda commented 2 months ago

Moreover, I intend to participate in a virtual meeting with Backdrop-CMS developers on Thursday.

I hope to record the part of the session which I present the matter of XMPP PubSub.

sjehuda commented 2 months ago

@Neustradamus has directed my attention to WP plugins and we have found a couple of plugins made by @Jehan

https://wordpress.org/plugins/jabber-feed/ https://wordpress.org/plugins/xmpp-auth/

The screenshots at https://plugins.trac.wordpress.org/browser/jabber-feed/trunk/screenshot-3.png indicate that the implementation is good!

A question might be arise: If this was made over a decade ago, and it does not seem to be active, then why should we do it now?

Because then there were no XMPP clients that supported PubSub as Movim and Libervia who are making actual use of PubSub and allowing to end-users (so called) to interact with PubSub.
Also, there was no concept of "social feed" which is a common practice in the recent 5 years.

From my impression, once PubSub is implemented by CMS and Wiki, I think that Gajim, Psi, DrDino and Kaidan will support PubSub too, not only Movim and Libervia.

I sense, that once XMPP developers will see that content generators put attention to PubSub, then XMPP developers will respond by implementing a better PubSub experience within 6 - 12 months.

Jehan commented 2 months ago

LOL I was wondering why I suddenly got notified for an XMPP-related topic after so many years. 😄

Anyway happy if these plug-ins get noticed finally after so many years. Back then, as a XSF member myself, I was trying to even get the XSF to use it on xmpp.org since it was running on Wordpress at the time! It never got much interest. I was the only person enthusiastic to be able to login on my blog without password thanks to XMPP, filter comments through XMPP (anti-spam, as people have to validate with a real XMPP address!) and publish feeds through XMPP (à-la RSS, but instant, better and without unnecessary polling!).

Back then, I was even envisioning a web through XMPP (xweb or something!) instead of HTTP (it was solving so many issues: at the time, there was no websockets or web push technology, just bland background-pull ajax; and we'd even have an integrated single login validation for every online resource, instead of "login to our website through this or that service"). That were the good ol' days where everything seemed possible for XMPP! 🤗

Anyway, not sure why you mentioned me now. I've not worked on these 2 plug-ins for years, I doubt they still work even (Wordpress evolved a lot since then!) and anyway I can't make any time for this anymore. I maintain other software and that's already taking most of my time! 😅

I still wish you good luck. Hoping things work out this time for XMPP syndication.

sjehuda commented 2 months ago

@Jehan Thank you for commenting on this matter and thank you for providing background on this subject.

I believe that Libervia and Movim (an some centralized IM platforms), that have given more emphasis to PubSub, have finally made this discussion possible again.

The most important things we need are:

Awareness of the mind: People to understand the potential of utilizing PubSub.
Tip involvement by publishing projects: Willingness of CMS and other content software (e.g. feed readers) to communicate with XMPP PubSub.
Deployment: PubSub to be used by people who use XMPP.

Once three of these conditions are fulfilled, we will see a new phase on XMPP (and probably resentment and trolling from anti-privacy agencies of sorts, as they want us to be relied on HTTP/HTML/JS).

Back then, I was even envisioning a web through XMPP (xweb or something!)

You are not the only one to think so, and I sense that we are getting closer to this day.

That were the good ol' days where everything seemed possible for XMPP!

Be ready for this day and we will be chatting about it over XMPP.

Anyway, not sure why you mentioned me now. I've not worked on these 2 plug-ins for years, I doubt they still work even (Wordpress evolved a lot since then!) and anyway I can't make any time for this anymore.

For you to be aware of my lobbying actions on this concern (there were over 20 projects to which I have sent this request).
For you to make recommendations, if you would.
For you to provide coding guidance concerning your plug-ins, if you would.

I still wish you good luck. Hoping things work out this time for XMPP syndication.

Thank you. It is a great honor to communicate to you.

P.S.

Off Topic I have made a [news bot](https://gitgud.io/sjehuda/slixfeed) (almost be ready for an official release) that can also download HTML pages as offline documents (no HTML browser is needed) and which also publishes items to PubSub. The reason for that bot, and [many](https://gitgud.io/sjehuda/slixmark) [other](https://gitgud.io/sjehuda/slixprint) bots soon to arrive (email, git-send-email), is to make XMPP more useful and also to keep people connected to XMPP, even when they have little human interaction. Most of my bots are designed to handle multiple JIDs at once - even 1000 - which is something that was not possible with previous Python libraries.

Jehan commented 2 months ago

For you to provide coding guidance concerning your plug-ins, if you would.

Not much to say. It was a very simplified approach, where I reimplemented myself the whole connection to XMPP (there were probably 2 reasons for this, though I don't remember enough to be sure: firstly, because that avoided having any "non-standard" requirement to my plug-in; secondly because anyway not sure there were any reliable PHP XMPP lib back then). I also ended up implementing SCRAM support in Auth_SASL2 PEAR package. Until then, it was simply impossible to have a fully compliant XMPP client in PHP because of this missing feature in base packages.

In the end, it was a nearly fully self-contained code (working with base bricks on a PHP server so that nearly anyone could install it, even when using shared hosting platforms where you usually have no say in what libs are installed), which was coded to do specifically what it was made for. It was very simple, just sending the needed data the correct way and able only to process the correct answers. It would not do more but it worked well for what it needed to do, I guess. 🙂

xenocrat commented 2 months ago

@Jehan: Thank you for taking the time to write those details. A lot of the reasoning you describe overlaps with the aims of this project, so I will definitely poke at your old code to help me consider how things could be done most simply and with the fewest dependencies.

sjehuda commented 2 months ago

I suggest you to ask "edhelas" from Movim about Moxl.

edhelas has said that Moxl is crafted specificially for Movim. We might want to use the projects of @Jehan.

edhelas at xmpp:xsf@muc.xmpp.org?join

Moxl is for Movim XMPP Layer, it's not meant to be an independant library and there's no plan for it.

moparisthebest

I like to think it's because the protocol is so simple that it's not worth writing a library for :)

They have also added this image to realize how easy it is.

sjehuda commented 2 months ago

There is an independent PHP library called xmpp-php which has not PubSub yet, but IQ support which might help.

xenocrat commented 2 months ago

Thanks. I have other priorities at the moment, but I am continuing to investigate. Ideally I would find an actively maintained library that I can use, because I don't want the extra burden of maintenance. It's not encouraging that all the PHP XMPP implementations I have found so far are either abandoned or archived, and the maintainer of the one actively maintained solution (Moxl) says that I should just write my own.

Without at least a well-maintained and known good solution to provide basic socket, TLS, and auth functionality for XMPP, I am unlikely to have time for writing and testing this feature. It would significantly change the situation if I could lean on a solid and actively maintained solution for that basic functionality. The Google PubSub library is maintained, but due to it being Composer-based and having an extensive dependency chain, it is very much incompatible with the aims of this project.

The glue code needed to actually implement PubSub syndication in Chyrp Lite would be quite simple due to Chyrp Lite's trigger-based architecture. In recent days I've made changes to Chyrp Lite's Atom feed generator that would allow the Atom entry to be generated easily when a post is published/updated.

It would really make a difference if there was a suitable and actively maintained XMPP library for me to build on; I suggest that as a focus for your efforts.

sjehuda commented 2 months ago

This is very embarrassing to me.

and the maintainer of the one actively maintained solution (Moxl) says that I should just write my own.

The maintainer, who is a wonderful man, has said that Moxl is intended for Movim and someone in the groupchat has sent the image in order to realize a current situation, albiet it was not absolutely true as there was a time when Gajim and PyTalk have shared the same library to handle XMPP.

Even the owner of selfoss has also made arguments about Moxl which made me to conclude that we have to have a well-maintained PHP library first.

Without at least a well-maintained and known good solution to provide basic socket, TLS, and auth functionality for XMPP, I am unlikely to have time for writing and testing this feature.

It would really make a difference if there was a suitable and actively maintained XMPP library for me to build on; I suggest that as a focus for your efforts.

I think you are correct.

xenocrat / chyrp-lite