Example of reading the data feeds

[DomsRoberts]

I have been writing a .Net feed reader using the library and apart from numerous end point failures, I'm having trouble getting the data back from RpdePages. Do you have example code on how to parse the data out of them? The use of Values to hold the data makes everything more convoluted and the use of new to override properties. I have a data item that has 3 different types of EndDate, 2 of them are Null and 1 has a value. How are you meant to use it to get the value? There are plenty of other issues like this.

[DomsRoberts]

I assume you have given up on this project when you realised how ludicrously overly complicated you have made it??

[nickevansuk]

Sorry for the delay in reply @DomsRoberts, the project was actually based on Schema.NET, and there’s some pretty clever stuff in there - hopefully the complexity is justified, but any suggestions for improvements welcome of course!

Check out SingleValues classes and you’ll find a few methods for accessing the values. My advice would be to cast events to the appropriate subtype and then access the properties, which ensures you don’t hit issues around multiple types of EndDate.

Happy to jump on a call tomorrow if that’s helpful? I’m aware that documentation for consumption of feeds is somewhat lacking (and will soon be addressed!) but in the mean time it might be easiest to just talk it through?

[nickevansuk]

Also I should mention there’s a vague ambition in the community to create an open source feed reader in .NET, PHP and Ruby - though no one has had time to do anything about this yet - contributions welcome!

[DomsRoberts]

If I look at the basic task of extracting a simple text representation of an Address from a ScheduledSession. I can't use the base type of Event and share the code to extract the Address from each of the types derived from Event as that is then the wrong instance of Location and will be empty. I take the ScheduledSession Location which is an instance of a Place. Then the Place has a Property called Location which is either a string or a PostalAddress. PostalAddress has some Properties that are the string representation of the address. But, it is derived from Schema.Net.PostalAddress, thus I might have to cast it as a Schema.Net.PostalAddress and that can have an AddressCountry that is either a string or an ICountry. Then I have to handle both of those cases then an ICountry implements IPlace and IThing and IAdministrativeArea and soon enough I am handling a vast number of possible ways to extract the text representation of an Address.

Looking through it, it seems that the way to progress is to keep everything as OpenActive types where possible and avoid using the Schema.Net base types at all costs. Otherwise the number of possible paths that need to be handled just explodes.

[nickevansuk]

The deserialiser will use OA .NET in preference to Schema.NET - so you don’t need to check the Schema.NET base types as data should always be in OA.NET.

The OpenActive modelling spec is based on schema.org, which means all schema properties are valid, but that OA tightly defines a subset of them.

Therefore, if the property you are trying to access is in the OA specs, you only need to worry about the OA classes.

Most OA data right now should exist within the OA classes and beta properties? Do you have examples where this isn’t the case?

Also Address should be the same overridden property for Event and its subtypes - otherwise this is a bug - as it is the same across all.

Handing Address as a string vs PostalAddress should be the only complexity you are dealing with here? (And there’s an effort to move all publishers towards the latter - the former exists primarily for backwards compatibility)

[nickevansuk]

There is a notable section the documentation of this library missing that covers deserialisation. I’d be keen that we write up the learnings from this thread into the README - all the things that you’d which you’d known when starting out here?

I appreciate that it must be frustrating without much of this context to figure it our with trial and error.

Some tools and other documents that might be helpful for you when using the data:

• https://visualiser.openactive.io/ will help you browse the data feeds that are most strictly OA compliant (a number of the feeds are using old versions of the OpenActive model - using the feeds available in this tool is the easiest way to start using OA data).
• https://github.com/openactive/developer-documentation/issues/35 is a proposal to improve the OA docs for data users (the first of many such proposals)
• https://www.getthedata.com/openactive is useful to understand which properties are in use across which feeds
• https://endpoint-analyser.openactive.io/ Is an old version of something similar

All thoughts welcome, and a call still on offer if that would be a high bandwidth alternative to filling in any remaining gaps in context that you might have

[nickevansuk]

@DomsRoberts how are you getting on with this?

[DomsRoberts]

Still incredibly frustrating when attempting to convert from the feed to a more simplistic model. I can understand the need for flexibility but I would put the anticipated development time, to create a workable reader that converts to a limited data set, to at least a month or two of full time development. Reading the first couple of data sets that respond and simply trying to extract the location, contact details and name of the activity leads to quite a complex amount of work as data can be in so many different places.

[nickevansuk]

Interesting @DomsRoberts - which datasets did you start with? Are you able to share your code at all?

[nickevansuk]

So I was just thinking a little more about this...

If you pick a SessionSeries feed or Event feed, with the exception of those within the "legendonlineservices.co.uk" domain (are not fully conformant, though the below would still work for them), so that's any from the dropdown in https://visualiser.openactive.io/

For these feeds, properties can live in the root SessionSeries or the superEvent, so you'd need a bit logic for extracting each property that handles that.

Thinking about what would be useful for the library, how about an extension method like this:

    public static class ExtensionExample
    {
        public static O GetPropertyWithInheritance<O>(this Event @event, Func<Event, O> accessor)
        {
            if (accessor == null) throw new ArgumentNullException(nameof(accessor));
            O AccessorWithNullCheck(Event e) => e == null ? default : accessor(e);
            return @event switch
            {
                SessionSeries e => AccessorWithNullCheck(e) ?? AccessorWithNullCheck(e?.SuperEvent),
                ScheduledSession e => AccessorWithNullCheck(e) ?? AccessorWithNullCheck(e?.SuperEvent.Object) ?? AccessorWithNullCheck(e?.SuperEvent.Object?.SuperEvent),
                Event e => AccessorWithNullCheck(e) ?? AccessorWithNullCheck(e?.SuperEvent),
                _ => throw new ArgumentException("This type is not yet supported"),
            };
        }
    }

This would allow you to do the following:

  SessionSeries sessionSeries;
  var locationName = sessionSeries.GetPropertyWithInheritance(x => x.Location?.Name);
  var locationContactTelephone = sessionSeries.GetPropertyWithInheritance(x => x.Location?.Telephone);
  var organizerEmail = sessionSeries.GetPropertyWithInheritance(x => x.Organizer?.Email);
  var organizerName = sessionSeries.GetPropertyWithInheritance(x => x.Organizer?.Name);
  var activityNames = sessionSeries.GetPropertyWithInheritance(x => x.Activity?.Select(x => x.PrefLabel));

Note using the above you don't need to cast the Event to SessionSeries first, so you can use the above straight from the RPDE feed.

Would that do what you need? As previously mentioned, very happy to have a call if at any point you think that would help accelerate the discussion here and ensure you have the support you need.

[nickevansuk]

Hey @DomsRoberts any thoughts on this?

[nickevansuk]

Hey @DomsRoberts how are you getting on with this?