Event channels

[joachim-n]

Problem: Connection to directories

When is something an event or service? Directories will usually have an organisation page for a Voluntary sector or community service provider, but then other pages for the services they provide. Sometimes these related pages are just one-off events or services that some people might expect to find in a calendar, such as holiday playschemes, breastfeeding cafes, dance classes etc.

For example:

https://www.lbhf.gov.uk/family-information-directory/addison-youth-club https://www.lbhf.gov.uk/family-information-directory/kidscapes-zap-workshops-9-16s-who-experience-bullying

This may be more of a governance and process issue than a tech one, but consideration is needed on linking events to directory pages or whether events can appear in directories.

Proposed solution

We looked at creating an event channel using the Directory content type and it seemed to work well. To do this we need to:

• [ ] Add the directory channels field to event content type.
• [ ] Event taxonomy needs to be moved to directory taxonomy way of doing things.
• [ ] Consider date filters (see below)

Switching to this new way

Calling the event channel Events will override any existing /events view but we might also want to provide documentation on how to switch to the events channel way.

How people look for events

We don't know how people look at events.

• By activity or by date
• On a calendar or in a list
• what's near me (location search)

If we did calendars then we need to be sure it works well at mobile sizes and it's easy to use.

[joachim-n]

We can put events into directory channels, but the directories module is very opinionated.

For instance, on a directory channel, it wants to show a specific hardcoded view, which shows a list of results.

How would we show a calendar instead?

[ekes]

We can put events into directory channels, but the directories module is very opinionated. For instance, on a directory channel, it wants to show a specific hardcoded view, which shows a list of results.

I was going to say 'and rightly so' to 'opinionated'. It was designed as on out-of-the-box no-configuration (or rather auto-configuration) search and filter system, that could have 'content-editor' facets. But in fact in the case of the 'specific hardcoded view' @Adnan-cds was also already talking about maybe wanting to change the view (in that case for postcode search for service areas - or something similar).

So maybe if there is a way of making the view configurable/swappable - while keeping the out-of-the-box no-configuration defaults - we could implement that in directories.

[joachim-n]

Opinionated isn't necessarily a bad thing! :)

Simplest way to make the view swappable would be to put a field for it on the Directory Channel page. Either to allow different displays of the one view, or selecting different views.

[ekes]

There is also all the autowiring (or rather just unwiring when you change) of facets and blocks. Facets 3 might make this easier, so looking at upgrading that might be a thing, it'll need to happen at some point. Factets 3 makes the facets into the view filters.

[joachim-n]

Ah, just seen that the localgov_directories_location submodule adds a display to the localgov_directory_channel view in localgov_directories_location_install(). So there's precedent!

[ekes]

True just for a display on the same view and that's already getting a bit brittle if I recall correctly. So making it a better pattern others could reuse would be nice.

[joachim-n]

I think the whole extra fields system is brittle -- I think it would be better done with computed fields, now that they have better support in core. There's the problem that bundle fields still have bad support with Views, but as these are all just output rather than computed data values, that's not a problem.

[ekes]

If switching the computed fields per bundle is now better I could see switching to using them might bring advantages.

[adds: the facets one isn't often used, and would be completely redundant if moving to facets 3.x I'd assume, as it'd either be filters in the view, or a block that gets placed by other rules]

[willguv]

Sounds very promising - thank you! Are there any product implications for this (eg unexpected changes for existing users of directories)

[joachim-n]

I'll file an issue on the directories repo for conversion to computed fields. For the same thing with events, there's #139.

[joachim-n]

The story so far, general brain dump!

Recurring dates data entry

The date_recur module combined with date_recur_modular will do pretty much what we want for complex recurring dates. We need to use one of the more complex widgets. We might want to look at how to allow the user to switch between widgets on the fly, depending on what kind of recurring date they are dealing with. I have no idea how that would work with Field API widgets!

Recurring dates in search results

date_recur writes its own table of date occurrences, with Views integration, and the current localgov_events uses that to make the list of events.

That means that currently, an event that is monthly for a year will show 12 times in the view.

I've looked at different ways of getting recurring dates into Search API:

• date_recur_sapi: (https://www.drupal.org/sandbox/sam/3200275) This puts multiple search API items into the search index. I stopped looking at it when @ekes pointed out date_occur to me, which includes something very similar.
• date_occur: (https://www.drupal.org/project/date_occur) This puts multiple search API items into the search index. It also has features beyond what we need, such as being able to add a node that overrides a particular instance of a recurring date. This is quite nice -- for example, you could override one occurrence if an event's location changes for just that one time. However, this handling is pretty complex and not bug-free.
• date_occur_computed: (https://www.drupal.org/sandbox/ekes/3427814) This exposes a computed field on the event node, which returns date occurrence values. Search API sees only a single item, but adding the computed field to the index makes it see a multi-valued date.

These all have problems!

date_occur

date_occur defines a custom SearchAPI datasource. This uses the date_recur to return multiple items to add to the search tracker.

When these items are added to the search index, the custom datasource provides items which are instances of a DataType plugin, not an entity. That means that none of the other fields on the entity are availble (facets, full text search, etc). It would take a fair amount of time to fix this.

One way would be to add a property to the plugin that lets Search API drill down into entity fields. However, I think that would be problematic in an index which also has Directory Pages in it, which are normal entities -- we'd have to have two copies of everything.

The other way I've thought of would be for the custom datasource to return items which look like the entity, but have a field with a particular occurrence of the date. This would work like this:

1. The custom datasource returns IDs which are composed of the entity ID + an identifier for the date occurrence (currently it uses the date string; @ekes said using the occurrence delta causes problems)
2. When the item is indexed, the datasource loads the entity, and then sets (but doesn't save!) a date value for the occurrence
3. Similarly, when the item is loaded, we return a doctored entity with a field value changed to the occurrence. (I have no idea whether this would actually work!!!)

date_occur_computed

This currently works well with Search API, however, Search API only gets the one event node indexed. So an event that happens every month for a year will only show once in a list of events for the year. Facets work for all the date occurrences, so if you used a facet to limit a particular month, it would show.

This is not the current behaviour of the localgov_events's events view.

It could be argued that this is clearer, however, we need to be aware that this is a change.

There's also the matter of sorting -- I've not looked into how events are sorted by date. For example, if I have a monthly event on the 20th and a one-off on March 10, and I filter by facet to March, the one-off should show first. It might however be that the monthly shows first because SearchAPI is using its first delta of Jan 20 as the sort value!

It also potentially won't work with a calendar, where we definitely want a copy of the event to show for every occurrence. Again, I've not tested this yet. There another issue where @ekes outlined the calendar modules, and it looks like all of them need further work.

Facets

Date facets

search_api module's own date facets work with date_occur_computed. I didn't get as far as getting them to work with date_occur.

They don't work the same as what localgov_events's events view currently shows.

Currently, we have a Views filter which shows a start and end date and a range dropdown. If views filters can work alongside the facets from the Directory system, we could keep using views filters.

The search_api module date facet shows a list of months (or years, or days -- granularity can be configured) for the current result. Furthermore, the date facet can be made hierarchical: selecting March for instance then shows all the dates in March which have results.

I think overall the current filtering is more intuitively what people expect from search dates, rather than the search_api date facet.

There are other date facet modules, but they all crash and need further work:

• https://www.drupal.org/project/facets_date_range
• https://www.drupal.org/project/facets_date_range_picker
• https://www.drupal.org/project/facet_granular_date (apparently the functionality is the same as what search_api now has)

Integration with Directory channels

The facet module determines whether to show a facet block by considering the current page, and whether that matches the facet source for the facet.

This association breaks with directory channels, because the facet source is an embedded view, and so the comparison of the routes fails. We'd need the same sort of handling for that the the directory facets do.

Integration with Directory facets

Directories use a single facet for all of the directory facet entities. That shows as a block, but there's also a FieldAPI 'extra field' that outputs it too. I'm not sure what that's for and whether it's still relevant, but we might need to make sure we provide that too.

Calendar

I've not got as far as looking at calendars. There's a summary of exploration work so far by other people at https://github.com/localgovdrupal/localgov_events/issues/140.

What we do with calendars influences which SearchAPI integration we use, and vice versa -- it depends on which integration options make each occurrence of an event show in the calendar.

Integration with directory channels

As in conversation with @ekes above, we need to figure out how to allow channel nodes to show a calendar instead of a list.

Next steps

We need to figure out whether we want multiple items or single items, and based on that, do further work on the revelant module for date_recur integration with SearchAPI.

This decision need not be final -- no data schemas are affected, so changing from one module to the other in a future release would mean changing config structures and re-indexing search data.

So one possibility if there is a deadline for an MVP might be:

1. Build an MVP with date_occur_computed, which allows a list of events in a directory channel.
2. For phase 2, convert to using date_occur to allow multiple events in a calendar.

[andybroomfield]

Interesting reading.

For reference, this is the module we use to power our event channels functionality https://github.com/bhccwebmaster/bhcc_events_plus

Example channel https://www.brighton-hove.gov.uk/libraries-leisure-and-arts/events-libraries/

Though this is still with the current events view so it's more of a hack. We had issues before trying to run events via directories as that uses the search API and didn't play well with recurring events, interested in some of the above though we want to move away from showing every event instance, and just the next one with the recuring rule. We also need to be able to modify the events vieww to be quite different and have very different filters, would the merge with directories achive this?

[ekes]

The most minor, but possibly really helpful integration of Events and Directories:- Bin day(s) https://drupal.community/@ekes/112183873888559501 https://toot.me.uk/@revk/112180343888447541

Search on address, get bin days events for your area. Bonus put it in your calendar - or whatever app - to remind you.

[willguv]

@andybroomfield could you tell me more about this please

We also need to be able to modify the events view to be quite different and have very different filters, would the merge with directories achive this?

[willguv]

Hi @joachim-n your suggestion for an MVP seems like a good one

“Build an MVP with date_occur_computed, which allows a list of events in a directory channel”

To know for sure it would be good to get T-shirt sizes for the various options. Please let me know if this is tricky to do. Thanks

I had a couple of questions about date_occur_computed

This currently works well with Search API, however, Search API only gets the one event node indexed. So an event that happens every month for a year will only show once in a list of events for the year.

• If a recurring monthly event in March, say, has expired, will the list will show the next event in April?
• If a user filters the list to show May events, will a recurring event show even if there's a recurrence in April?
• Could the event page show all the dates that an event is happening?
• Could the list page show the date and number of occurrences to indicate the user there’s more than one?

Happy to pick this up in a meeting if easier

[andybroomfield]

@willguv We going to need to add an age range filter as a standard views filter (minimum and maximum) and apply that to the events and our events channels view. They may be more. Ideally there are some things we would like on the events and events channels views that we wouldn't want on the main directories one. That raises another question, will the /events view then still be avalible to show all events, as it sounds like thats intended to be a channel in it's own right. I think we're still going to need an all events view.

I also have another question, what happens if an editor puts a standard directory entry into an events channel or vice versa? Will the channel itself have to have some flag to set it's behaviour as an events channel?

In reference to the above

Presumably when a recurring monthly event in March, say, has expired, the list will show the next event in April?

Thats going to be a need for us, although we're looking at only show the event once in the listing to stop daily events swamping the whole list. However it's important that date filters that apply to events also pick up the events occurance.

[andybroomfield]

Ref #54 and https://github.com/localgovdrupal/localgov_directories/issues/166 which is where this came from for BHCC. Raises another question, how possible would it be to use facets fully outside directories?

[joachim-n]

If a recurring monthly event in March, say, has expired, will the list will show the next event in April?

Yes. A manual test confirms that the first date in the future is the one that shows.

If a user filters the list to show May events, will a recurring event show even if there's a recurrence in April?

Yes, BUT. My current (admittedly very basic build) has that event displaying its April date, even though I am filtering to May. This might actually be the critical flaw that tanks this particular implementation -- the displayed date is coming from the rendered entity, and thus the recurring date field formatter. This is doing its best and showing 1 occurrence, as configured, and showing the first occurrence from the present time. HOWEVER, this formatter has NO WAY of knowing that the current View is being filtered to show May! Obviously, this looks like a total WTF to the user!

I think this suggests we should go down the route of indexing an item for each recurring date, rather than using the multiple-value approach.

Unless @ekes has any thoughts about how to make the recurring date field formatter aware of facets? But it seems to me that it's going to be pretty messy -- two totally unconnected systems trying to talk to each other. (Then again, the whole facets system is based on things happening in one block reaching and fiddling with the query in the main content...)

Could the event page show all the dates that an event is happening?

Do you mean the page for a single node, or a directory channel? And do you mean a list of the dates in one result item, or multiple items?

Anywhere an event node is shown, we can show all of the dates, by configuring the field formatter. (But, see above for a limitation!)

Could the list page show the date and number of occurrences to indicate the user there’s more than one?

Not currently, but I imagine the recurring date formatter could be extended to show how many occurrences are in the future (for ending events) or how many occurrences are in the next year (or other configurable period, for non-ending events).

[ekes]

Unless @ekes has any thoughts about how to make the recurring date field formatter aware of facets?

It'd have to be at least switching from rendered entities to fields in the view, but even then the field would have to do quite some lifting, getting the information from the facet... ok... maybe with facets 3 (they're more wired into views then), but even still that plus the logic on what to then show. I'm not sure.

Could the list page show the date and number of occurrences to indicate the user there’s more than one?

One thing we've done with other sites - and I don't know if you can make a good generic solution from this - is show the machine generate repeat rule, it is often good enough; but with an additional text field for a human to describe the repeat which will override the machine generated one, for the occasions where the machine description is horrid. Works, but only for editorial sites with trained staff.

[joachim-n]

I think the way forward now is to look at getting distinct items into SearchAPI for each occurence of a recurring date, and have these items behave as entities.

This is different to what date_occur_computed does, because that does only one item. It's different to date_occur, because that puts in a custom data object which doesn't work with fields.

I'm not sure yet how to do it or whether it's even possible (@ekes feel free to tell me this is insane :) ), but I think it gives us the best base for SearchAPI to then do useful things like calendars and so on.

I think it's also the first thing to tackle, as most other things depend on how SearchAPI data is structured and made available to Views.

I'll file a new issue for that work specifically.

The recurring event widgets topic is now at [#145].

There's also the topic of making event chanels able to use different views or view displays -- not sure if there's an issue for that already.

[ekes]

Search API is really just leveraging Entity API, but mainly Typed Data API. So with the date_occur it's just Typed Data, which means some of the processors, working with entities, don't work. Drilling down through the fields does work - as they're Typed Data. I assume the issue then is the processors?

• Like the one reported as a bug in the module issue queue where Search API doesn't check type before accessing a field;
• or the inability to render the entity directly in a display mode.

Without creating an Entity API entity I'm not sure how you'd make it operate like one, display modes and all. It might be easier, if less neat and tidy, to continue with a Type Data object but write processors (they're pretty easy plug-ins to write). Taking the second example above to get the entity in a display mode the processor would be able to render the referenced entity in a display mode.

[joachim-n]

It's not just the processors, but also the fields, I think? And I think the problems with writing processors and fields are that:

a. We're going to need to duplicate every processor and field we use, or might want to use b. We're going to need to add both the original AND the duplicate to the search index, because the directory search index also indexes regular nodes. Which is going to get really messy.

I was thinking we could do something like index the event node ID + an identifier for the recurring date, and when SearchAPI loads that item, we load the original event node, and either doctor its field values, or use a computed field that return a single date value, so that SearchAPI sees that event as being on only one occurrence.

[ekes]

Fields should work, this is from the Typed Data API level. You should be able to drill from a reference field into an entity into its fields into any entities it references etc. There isn't even a way of writing custom fields for Search API those are in fact processors. Processors work too, just not the ones that assume they are working on an Entity at the top level indexed item (with a Drupal Entity API capital E).

add both the original AND the duplicate to the search index

In the sense that the referenced entity would be indexed with the typed data object yes. So it will index the same referenced entity multiple times for each object, each repeat, but that's kinda the point. The alternative being to index the occurrences on a single item.

the directory search index also indexes regular nodes

If you really want them in the directories, then yes. Search API does suprisingly well at mixing types in the same index. But it is worth avoiding if possible.

index the event node ID + an identifier for the recurring date, and when SearchAPI loads that item, we load the original event node,

That's exactly what you are doing when you index a date_occur object without including the referenced entity. So that's the default behaviour in effect. Expanding that: You can't index it as a node because a node is indexed use the entity:entity_type:id search api item_id, works because they are unique. So the date_occur object has its own unique search api item_id for the index item based on the entity identifier and occurrence. After that it has a field to for the node ID, and for the date.

[andybroomfield]

A little experiment in case it is useful: https://github.com/andybroomfield/localgov_events_date_search_api/

This is a basic proof of concept using two Search api provided event subscribers to index all dates into the events search index and sort by the searched for date. Still only a single instance of an event returned.

[joachim-n]

I've got the SearchAPI part of this working -- https://www.drupal.org/project/date_recur_search_api

However, I'm a bit stuck on the best way to fit this into events and directories. Basically, we have to break something, and we have to decide what we break.

My first thought was to combine everything into one index -- put event nodes into the localgov_directories_index_default alongside directory pages and other directory content. I even wrote a small SearchAPI helper module so that we could use common SearchAPI fields across the two different datasource types (https://www.drupal.org/project/search_api_common_field) so that Views and other things consuming this index would not need to be changed.

However, having a second datasource in the index for event nodes would mean that localgov_directories_field_config_insert() and ConfigurationHelper would need a major rewrite. This code reacts to when a node type gets any of the directory-related fields added to it (directory channel / facets / title sort) and automatically updates the index to include this node type. ConfigurationHelper expects to work with the entity:node datasource, but with events nodes we use the date occurrences datasource instead. So we'd need some kind of system for node types to declare which datasource they should use, or for a way for datasources to announce that they're the one to handle a node type.

That seemed rather complicated, so I had a rethink, and I considered: do we actually need to intermingle date occurrences and other content? Arguably, if you're showing recurring events, you have some sort of a date-based listing or calendar, and nodes which have no date don't belong in this. Conversely, if you're listing non-date things, you don't want hundreds of event occurences in the results -- if you really want to list an event in there, say, because you want to show all things in a certain location, then you probably want the plain version of the event, that is, the single node, which you'd get from the entity:node datasource. In that case, it makes sense to allow plain event nodes to be in the existing directory index, and make a second index for event occurrences.

So, we make a second index, and we need a different view to use that index (but we needed directory channels to allow selecting the view anyway, so we could have calendar views). But then we have a different problem: the localgov_directory_channel_types no longer makes complete sense. You could create a Directory channel and for its localgov_directory_channel_types value select 'Directory page' and 'Event', but if you then selected the events view, you would see no 'Directory page' nodes (since they can't be in that view's index), and if you created a 'Directory page' and set it to show in that channel, it would have no effect.

I'm a bit stuck at this point! Any thoughts?

[andybroomfield]

Do we actually need to mix directory entries and events? The BHCC events Channel feature reuses the facets field but directories and events are conceptually separate things.

[andybroomfield]

Am testing date_recur_search_api with the BHCC events, its working good so far in that I can see date instances and the computed field.

I'm having an issue getting the facets block working as checkboxes, I wonder if that is to do with using the date_recur search datasource.

[andybroomfield]

A follow up, I'm seeing that when the facets are passed through the hook_preprocess_facets_item_list i'm seeing the following for events

array:18 [▼
  "facet" => 
Drupal\facets\Entity
\
Facet {[#2677 ▶](http://bhcclocalgov.test/events/search#sf-dump-940268222-ref22677)}
  "items" => array:144 [▼
    29 => array:5 [▼
      "#type" => "link"
      "#url" => 
Drupal\Core
\
Url {[#7947](http://bhcclocalgov.test/events/search#sf-dump-940268222-ref27947) …12}
      "#title" => array:7 [ …7]
      "#wrapper_attributes" => array:1 [ …1]
      "#attributes" => array:3 [ …3]
    ]

but with directories I see

array:18 [▼
  "facet" => 
Drupal\facets\Entity
\
Facet {[#2677 ▶](http://bhcclocalgov.test/events/search#sf-dump-115461196-ref22677)}
  "items" => array:144 [▼
    29 => array:2 [▼
      "value" => array:5 [ …5]
      "attributes" => 
Drupal\Core\Template
\
Attribute {#7297 …1}
    ]

[andybroomfield]

@joachim-n I'm convinced its something to do with the module weighting of theme_preprocess_facets_item_list as I can get them to display when modifying the directories version in localgov_directories_preprocess_facets_item_list but not in our custom one in bhcc_events_plus_preprocess_facets_item_list which I assume needs to be run after facets_preprocess_facets_item_list

[joachim-n]

Discussed this with @rupertj at the (occasionally weekly) events meeting.

Conclusions:

• Don't mix events in the same SearchAPI index as non-events. Create a 2nd index in the directories server for events
• Node bundles can optionally declare in their bundle info:
  • which index they use
  • which view they default to for search result lists
• Add UI to the node type form for setting this. Store this in the node type entity's 3rd party settings, then set in the bundle info (so this can be done both in code and config)
• ConfigurationHelper::getIndex() uses the index from the node type, with Constants::DEFAULT_INDEX now as a fallback
• UI to choose the View to use for a directory channel, with the view declared in the node bundle info presented as the default
• The localgov_directory_channel_types field needs to segregate the options, because you can't put node types from different indexes in the same Directory channel.
  • The localgov_directories_entry_types selection handler needs to validate that you don't pick incompatible entity types.
  • Add widget alter to use JS states to disable incompatible options when one is selected. E.g. select 'directory page' and 'event' becomes disabled
• Facets - no idea yet!

The limitations this imposes would be:

• You can show event nodes in the same Directory channel as non-event notes

@willguv @ekes