Automatically re-index groups & members on change

[modelm]

Presently groups & members are only indexed when the indexing cronjob runs. If a group or member profile is edited or deleted, nothing happens to the corresponding Elasticsearch index until the indexing cronjob runs again.

Write functions to re-index individual groups & members and hook them to the appropriate BP actions.

[nickchomey]

@mikethicke @modelm I just discovered this repo and mentioned it in the official ElasticPress-BuddyPress integration issue https://github.com/10up/ElasticPress/issues/3.

As I said there, I'm eager to make this full-featured. So, if you could give me some guidance, I'd be happy to do the legwork. Any thoughts?

[mikethicke]

@nickchomey We haven't done work on this plugin for quite a while other than minor changes to maintain compatibility with newer versions of ElasticPress and our setup. The plugin was developed before my time here so I don't know it that well and the original developer is no longer part of the team.

I think this work could serve as the basis for a general purpose adapter for ElasticPress to Buddypress sites, but there is code specific to our setup that would have to be removed or refactored.

I don't have the time to work on actively developing this plugin at the moment, but I would be happy to review Pull Requests or help troubleshoot.

[nickchomey]

Thanks very much for the quick reply!

That's perfectly fine that you don't have time to actively develop this - I'm quite happy to do the work myself.

Ultimately I will need to browse all the code to get an idea of how it works and tinker around, but would it be possible for you to give a general summary of whatever you think is relevant to such a task to "modernize" the plugin, make it useful for a general user, and expand its functionality to index other buddypress/buddyboss tables (e.g. bp_activity)?

For example, which things are specific to your setup?

Are you aware of anything that is out of date/not using the latest ElasticPress mechanisms?

What would be your general approach to automatically re-index Bp data?

Would you do anything differently if given the chance to start over?

Perhaps it would be best to create/update any issues for whatever comes to mind and I'll start working away on it all?

Thanks again!

[mikethicke]

Hey Nick---all this is off the top of my head, so take it for what it's worth (not much).

• We reference a specific post type---humcore_deposit---in one place to deal with a quirk in our site. It would be better to have filter hook here and move that code to a plugin particular to our site.
• We have code that iterates through multiple WordPress networks, as our setup is a multi-network WordPress install and we have a custom database structure. It's possible this code is ok, but I would want to verify that it works in a single site or single network install.
• We're doing some other processing and filtering of results in the plugin. For general use, I think it would make more sense to use filters and move that processing elsewhere.
• I think elasticpress-buddypress.js probably either needs a full rewrite in modern JS / React, or eschew javascript entirely and do all the rendering in php.
• I would take a look at how ElasticPress triggers indexing when a normal post or page is published / updated. I believe that EP indexes incrementally, so you shouldn't need to do a full reindex in the normal course of operations. When to index a group, activity, etc is going to depend on what is ultimately getting stored in ElasticSearch.
• If I were to write this plugin for scratch, I would look to narrow the scope of the plugin as much as possible. I would ditch the frontend functionality and REST API and instead focus on adding BuddyPress types to the indexing and search. What you really want is ElasticPress, except it also searches BuddyPress items. This plugin, imo, tries to do too much.

[nickchomey]

Thanks very much! That's all very helpful, and I agree strongly with the final two points in particular. It seems like it would be prudent to mostly write this from scratch due to various legacy mechanisms and other extraneous things that are specific to your site (and WP Multisites in general).

I'll start poking around and try get a grasp of it all, as well as inspect the core EP features to see how it handles Indexables and incremental indexing. Hopefully I can port whatever BP mechanisms are used here into a copy of one of the core-EP Features (Woocommerce?). Whatever the case, this gives me a huge head start.

A couple more questions though, if you don't mind:

1. Could you elaborate just a bit on "the frontend functionality"? Do you mean the facets? Is that what elasticpress-buddypress.js is for?
2. The search bar on HCommons doesn't offer any live filtering when I type in it. Is the REST API function broken or is there any other reason it isn't working/being used? Too costly?

Both facets and live filtering seem like very useful things to have, so it would be good to know if there's any foibles with them. Of course, I'll leave those for a later day - the core functionality of indexing and searching BP tables is the important part.

Thanks again!

[mikethicke]

On (1) - elastic-buddypress.js generates search results by querying the /wp-json/epr/v1/query endpoint created by elasticpress-rest.php. It is enqueued on the search results page and injects content into the #content div. I'm not sure why it was done this way, but if I was writing the plugin I would look to drop all of this functionality.

On (2) - This plugin was written a number of years ago. The search bar on the site is just a static html form. When you're on the search page it does update the search results as you type, but that functionality is part of elasticpress-buddypress.js and only enqueued on the search page itself. I don't know the thinking of the original developers as to why it was implemented this way, though I agree live filtering from the search box itself would be how I would do it today.

Faceted search is a really good feature as you add different types of content. This does appear to be a feature of the ElasticPress plugin already, so I think I would look at how to integrate BuddyPress types into that.

[nickchomey]

Thanks very much yet again!

I'll leave you alone for now and get to work understanding and rewriting this feature. It would be impossible for me to do from scratch as I'm quite new to all of this, but with this plugin and the core EP features as examples, I'm sure I can get something to work in the coming weeks. Projects like these are the best way to learn, so I'm eager to get started!