Volunteering to replace and maintain

[preaction]

A couple months ago I created a proof-of-concept community blog site using my Yancy CMS. I would love to use it to replace MT in the current blogs.perl.org.

Features it currently has:

• User dashboard for managing / authoring blog posts
• Admin dashboard for managing users, comments, and blog posts
• Main page integrating all users' blog posts
• Rendering blog content from CommonMark to HTML
• User comments on blog posts (can be disabled on a per-post basis)
• User reactions (emoji responses) to blog posts

Features it would need before go-live:

• Explicit Twitter, Google, Facebook, OpenID Connect, etc... auth plugins
  • Yancy has an OAuth2 auth plugin, but it would be more useful to automatically get user information from the API of the provider
• Anti-spam features
  • Either don't allow direct account creation or require new accounts' content to be approved manually for the first week
  • Configurable anti-spam detection (via an existing tool?)

Other features that would be nice-to-have:

• User reactions (emoji) on comments
• Blog author comment management: Require approval, delete comments, edit comments
• User comment management: Edit (with change tracking), delete
• Blog post change tracking
• Arbitrary user pages: MT allows users to create pages that aren't blog posts, this would be trivial to add

I could host a preview on a cpantesters machine / domain for evaluation if this sounds interesting. Are there any other things that a replacement would need to do?

[mohawk2]

I would love to help with this. @ap What do you think?

[tobyink]

Other things that seem important to me as an end user:

• will you be able to import all the existing content?
• will this break links?

[preaction]

Unless and until I get access to the existing database, that can't be known. I see no reason why I couldn't, but also I seem to recall that one of the (early) replacement projects got hung up on that specific requirement.

The URLs seem easy enough to keep: Everything real is under /users/:username, and the intention is to continue that. The PoC uses almost the same URL structure, and that could be changed to be the same, or different, depending on what we end up doing with those posts.

The main problem with importing content will be how the content is stored. MT supports multiple types of input, and I don't know how it gets stored (I assume as the direct input with another column for the input type). If it can all be made into Markdown (specifically CommonMark), that'll work, but otherwise it'd need pandoc to translate. Without a translation, we'd have only HTML. In the short term it may be best to simply export the HTML of the original site's posts and host them as static files (perhaps from a Git repo to allow PRs if absolutely necessary). In the long-term, we can improve the translation project, and/or provide an HTML editor (WYSIWYG or otherwise).

[arc]

Doug Bell notifications@github.com wrote:

MT supports multiple types of input, and I don't know how it gets stored (I assume as the direct input with another column for the input type). If it can all be made into Markdown (specifically CommonMark), that'll work,

One of the available formats is raw HTML. The existing sidecar handles the multi-format stuff:

https://github.com/blogs-perl-org/bpo-sidecar/blob/master/lib/BPO/Convert.pm

That goes only to HTML, but I can’t imagine using Pandoc would be much harder. -- Sent from my phone — please excuse brevity

[preaction]

Doug Bell notifications@github.com wrote:

MT supports multiple types of input, and I don't know how it gets stored (I assume as the direct input with another column for the input type). If it can all be made into Markdown (specifically CommonMark), that'll work, The existing sidecar handles the multi-format stuff: https://github.com/blogs-perl-org/bpo-sidecar/blob/master/lib/BPO/Convert.pm That goes only to HTML, but I can’t imagine using Pandoc would be much harder.

Thanks! This could be used to get everything as HTML, and then Pandoc could be used to make HTML into CommonMark. Easy-peasy.

[ap]

@tobyink:

Other things that seem important to me as an end user:

• will you be able to import all the existing content?
• will this break links?

I won’t go with any replacement project that breaks links or loses any content. By that I mean not just the most obvious URLs but basically any URL that has had traffic according to the logs.

[ap]

@preaction:

A couple months ago I created a proof-of-concept community blog site using my Yancy CMS.

Mojolicious. 😐 I’m not categorically opposed, but I will confess I am disinclined.

I wasn’t sure how to respond at first and had to think about what sort of criteria are important to me for a replacement project – there have been too many attempts already. Then I got buried at work and never got back to you. Sorry. Anyway, one of the conclusions I did come to is that it would need to be something I can hack on whenever necessary, including taking over the whole project in case your time (or that of whoever else volunteers a replacement project) dries up. Which unfortunately makes a replacement subject not just to my values but at least to some extent also my personal tastes…

Recently I’ve been going over the user database, which is full of spam; I’ve known it’s there for a long time, but didn’t realise just how much it is. That’s a thankless job with the stock MT UI – which is why nobody was looking (or even realising it) –, but I’ve been finding that butchering MT as needed to get functionality in there in the quick&dirtiest possible way is not too painful. Once you’ve fallen out of all support and upgrade paths and no longer try to treat it as an extensible system with ambitions of a plugin architecture, it gets a lot simpler to coerce desired functionality into its guts…

That’s the kind of thing I care about – being able to do the community janitorial work behind the scenes, and being able to quickly ad-hoc any tooling I need to make that doable.

The PearlBee-based replacement project already exists as well, was reasonably far advanced, and the code for that is lying around somewhere. I want to evaluate that as well.

I could host a preview on a cpantesters machine / domain for evaluation if this sounds interesting.

How hard is it to set up? If I can set it up myself, that would be ideal. For a first look at it that would not be crucial if you simply haven’t streamlined the process yet (although that would have to be a goal if it is to become the replacement) – but if it’s already simple, so much the better.

[preaction]

I could host a preview on a cpantesters machine / domain for evaluation if this sounds interesting.

How hard is it to set up? If I can set it up myself, that would be ideal. For a first look at it that would not be crucial if you simply haven’t streamlined the process yet (although that would have to be a goal if it is to become the replacement) – but if it’s already simple, so much the better.

If you can use docker-compose, it's as easy as docker-compose up, waiting a bit for Postgres to load and initialize and the app to deploy its database, and then visiting http://127.0.0.1:3000. This README will help set up the initial admin user with a command.

If not, I'd need to add some more environment variables / configuration to be able to decide what database to use, and maybe a Make target for installing the OS-level prereqs and doing carton install, but that'd be about it.

[ap]

Docker will do for looking at it. Is the code available?

[preaction]

Yep: The Github repo has all the configs, the myapp.pl file has the code, all of it (the single file will likely be refactored into a set of modules with proper MVC).

[wbraswell]

@ap & @preaction

What are our next steps to move forward on this in some way?

What is the current consideration of continuing the existing PearlBee work done by André Walker @andrewalker ? https://github.com/andrewalker/PearlBee

What is the current consideration of allowing Henry Van Styn @vanstyn to use his RapidApp framework, as he has offered to do in the past? https://github.com/vanstyn/RapidApp

If all else fails, then I will volunteer to sponsor development myself using any of the current options, relying upon the expertise of current Catalyst maintainer John Napiorkowski @jjn1056 .

[ap]

That’s quite a list, isn’t it? Who knew there are so many volunteers for writing a blog engine. 🙂

I did mean to look at every one of those abandoned projects when the time comes, as well as consider anything else offered in the meantime, but more broadly my thoughts are basically this and this.

Keeping the lights on is taken care of obviously. After that I attacked the spam accumulated in places away from the front page over a decade of inattention. Some of that was building myself amenities to find the stuff and keep an eye on incoming content, but mostly just weeks and weeks of shovelling out the mountain of manure (really it was compost by this time…). For the sake of cleanliness, of course, but I am also extra uninterested in preserving all this garbage through a migration. [Ironically the site is probably gaining value to spammers now that it’s spick and span because cleanliness should also yield more Google juice. —Ed.]

Next on my list is more boring tedious stuff:

• moving to cheaper (and faster) hosting (and getting and an OS not old enough to go to school)
• SSL

Once the plumbing is all in good shape then I’m willing to think about the fun stuff.

[mohawk2]

I think it might help if you asked for and accepted volunteers to help. You refer on the Reddit to single points of failure; are you one?

[wbraswell]

@ap Okay so just to make sure I understand, you want to work now on moving to a new server & enabling SSL, after which we can discuss plans for moving forward with a more permanent upgrade solution of some kind?

[vanstyn]

Hi all,

Since I was mentioned in this thread, I just want to let everyone know that yes, I am still willing to build and host a new blogs.perl.org using the RapidApp-based Rapi::Blog should the powers at be decide to request it. I have already done this for the TPF blog, which my firm hosts for free. I have the resources to do the same for blogs.perl.org, and also wouldn't require any grant money/funding. This would however require additional Rapi::Blog development work to add required features (such as multi-user tenant blogs) so if I were asked to do this there would be a lead-time involved for me to find the time to write the new stuff, but no monetary cost.

I'm generally pretty busy so I don't really have much time to be involved in the community and voice opinions, etc. I'm ambivalent to whatever decision is made. But, if it is decided that this option should be explored, all that is needed is to ask. Thanks!

[wbraswell]

@ap I believe the option offered by @vanstyn may be our best long-term solution.

[wbraswell]

Of course, I think the ideas put forth by @preaction should still be exercised, if appropriate. We can use all the eyeballs and brains we can get! :-)

[ap]

@mohawk2:

You refer on the Reddit to single points of failure

You have me confused with someone else.

[mohawk2]

@ap - woops! Sorry about that. Nevertheless, that person's point was important and given none of us are getting any younger, I hope there is something like a succession (and contributor-increasing) plan in place. Anyone who's interested in keeping b.p.o going ought to be willing to help with the unglamorous cleanup work. Have you asked for help?

Also, I see this repo is under blogs-perl-org - would it make more sense to think about bringing it within TPF? Would that cause issues?

@vanstyn I see the TPF blog is on https://github.com/RapidApp/tpf-blog-live, with a few open issues. As a point of governance, should that live under the TPF GitHub organisation proper? Are you the only person with push access? I'm keen to avoid single points of failure.

@preaction Any code or ideas we can borrow from the above to make a Yancy solution even more composable and better? :-)

[vanstyn]

@mohawk2 - I completely agree, I think it should live under a TPF org rather than under the 'RapidApp' GitHub org, where yes, only myself and a few other non-TPF folks have push access (no one has asked). I'd be thrilled to transfer ownership and move the repo to a more official home, and to have any and all help offered.

Yes, there are indeed a few outstanding items with the TPF blog, but nothing critical currently. I will be addressing everything, it is just a matter of finding the tuits. I'll readily admit that the heavy reliance on me personally is rightfully an area of concern decision makers should consider when evaluating this option. In certain ways, I'm a one-man show with Rapi::Blog right now, but it doesn't need to be this way. It is important to note that the code is fully open and written with accessibility and welcoming new contributors in mind. Obviously, I would love for more people to get involved, and that hope is a factor in my motivation to offer to do this.

Rapi::Blog is just one of many applications built on the RapidApp platform. While it may not be very well known publicly, RapidApp has been under continuous development for over a decade, there are numerous mission-critical, enterprise applications in the world using it, and it provides the primary source of income for a number of perl devs in the community right now. While many perl jobs are contracting and disappearing, we continue to grow and be commercially successful, and that is the reason I have the resources to be able to offer this without need of grant funding. I would love to have more contributors from the community, but even if that doesn't happen, if I agree to do it, I'll do it and keep doing it. I would take that responsibility extremely seriously, as I do with everything I give my word and commit to doing. My firm, IntelliTree, has been in business for 20 years and we're not going anywhere.

[wbraswell]

@ap it looks like we have a winner in @vanstyn

[preaction]

My goal is that blogs.perl.org is moved to a platform that doesn't prevent a significant fraction of the users from using it, and that whoever volunteers to do that actually ends up following through. If that's me, I need to know what my next step is. If it's someone else, I really don't want to wait for a few years until we decide the rewrite isn't happening and we're right back here again.

[wbraswell]

@preaction & @vanstyn How can we all work together on this upgrade, so that we can move toward the formation of a functioning team instead of independent individuals?

[preaction]

From what I can tell, there are two parts that could be independently developed: The web app and the data conversion. There'd need to be a schema mostly-completed before that, but I suspect there already are (both for my PoC and Rapi::Blog).

[wbraswell]

@preaction So does that mean you could work on the data conversion and @vanstyn could work on the web app?

[mohawk2]

@ap What is the state of play here?

[wbraswell]

@preaction & @vanstyn Are y'all still both willing to work on the two independent tasks of data conversion (Doug) & web app (Henry)? If so, may I volunteer to set up a conference call to get the ball rolling?

[vanstyn]

I'm still game

[wbraswell]

@preaction Would you still like to team up? :-)

[Getty]

Just wanted to add up again, that 7 years ago I made the DuckDuckGo Community Platform with specific this feature to have blogs per user and commenting for that and everything with more datatypes, complete forum, translation platform even, everything in there. Its easily rebranded and all opensource. I could again get that old state out and quick rebrand it, and we would have a ready to go app, these days i can actually already also prepare a docker for it. The adaption requirement are minimal, its all already Catalyst, DBIC, good structure, Dziled, practical made with making it a learning experience for perl developers. When I have a tuit i will prepare something now (given that this topic gets on the table again after 7 years haha)

[mohawk2]

@Getty Is there any code that you could link to here if people wanted to take a look?

[Getty]

@Getty Is there any code that you could link to here if people wanted to take a look?

Something around this commit https://github.com/duckduckgo/community-platform/tree/81059e1cedca8ec5f21f5e9a8f3d8366778620b3 but as said, i will make a clean version of it.

[benkasminbullock]

Replacing one thing with another thing which isn't finished - what could possibly go wrong here?