Roadmap of SyndicationFeed

zhenlan commented 7 years ago

SyndicationFeed on .NET Core

While we are making System.ServiceModel.Syndication available on .NET Core, we are also looking into improvements in following areas.

[x] Custom Parsers SyndicationFeed will have a default parser for RSS and Atom feeds, but now we will give users the opportunity to use their own parsing function for the known types, for example, if a user knows s/he is going to use an exact date format which might not be supported by the default date parser, the user will be able to override the default parser with her/his own code.
[x] Date Parsing Many users have experienced that the original parser failed to parse some exact date formats, the default date parser will be improved to support more date formats.
[x] Optional Items in RSS/ATOM spec There are currently some optional items that the feed object model doesn’t have. We plan to add them and make the user’s interaction with the object easier.
[ ] Async support Current APIs to read/write a SyndicationFeed document are synchronous. A thread will be blocked until the whole document is read/written. These APIs will be changed to be asynchronous so a thread will not be blocked for I/O anymore. When the asynchronous call returns, a user has full document for further operation. Update (12/15/2017): After a lot of prototyping and testing, we found it was hard to provide async support while maintaining high compatibility with full .NET Framework. Since the main goal of this library is to provide compatibility so your libraries built on full .NET Framework can just work on .NET Core with little/no changes, we decide not to support async in this library. We recommend to use the new Microsoft.SyndicationFeed.ReaderWriter library, which has async design in mind from the beginning.

A new syndication feed

While we are improving on the existing SyndicationFeed, which gives users best compatibility for porting existing code from .NET Framework to .NET Core, we are investing in a new syndication feed reader/writer, which is expected to suit better for modern programming when users are writing new code.

[x] New syndication feed reader/writer Unlike the async support for existing SyndicationFeed where an async call will not return until the whole document is read/written. A new syndication feed reader/writer will support reading/writing on demand. It does not save the whole content in memory and it only reads/writes forward (similar to the concept of XML reader). The new feed reader/writer will have graceful error handling, extensibility model and other improvements we have been doing for the existing SyndicationFeed. It will be a separate package that will give maximum flexibility to users (eg. migrating existing code vs. writing new code).

Feedback

Both the improved SyndicationFeed and the new syndication feed reader/writer will be built on .NET Standard so they can be used anywhere .NET Standard is supported (yes, you can even use them on full .NET Framework). Both projects are still working in progress, but we'd love to hear what you think as your feedback is crucial for us to make it right for you.

RehanSaeed commented 7 years ago

jsonfeed.org has sprung into existence since I raised the original issue https://github.com/dotnet/wcf/issues/76. It would be nice to be able to support that too, since Atom and RSS are a decade old format with a lot of problems and undefined behaviour.

dodyg commented 7 years ago

jsonfeed is too new to be supported. There are so many attempts to JSONify syndication format and so far nothing catches on.

dodyg commented 7 years ago

The new feed reader/writer will have graceful error handling, extensibility model and other improvements we have been doing for the existing SyndicationFeed

What is the model in handling namespaced extensions?

Will the new library (Microsoft.SyndicationFeed) helps with format detection?

RehanSaeed commented 7 years ago

@dodyg It's the chicken and egg problem, if frameworks don't support it, it won't get popular and frameworks won't support it. According to Google Trends, jsonfeed.org is on par with Atom feeds in terms of interest.

Agreed that it might be too soon but @zhenlan's post mentions extensibility, a pluggable architecture so we could add a reader/writer for JSON should be considered.

zhenlan commented 7 years ago

I hope this example will show some ideas of the extensibility model of the new syndication feed reader. Every time reader.Read() is called it will return the next syndication element. If it is a "known" element type, you can use one of built-in parsers; otherwise, it can be parsed as ISyndicationContent, which will contain raw Xml content among other things. Users can also pass in their own implementation of ISyndicationFeedFormatter when instantiating the reader for custom parsing.

Current implementation of syndication reader and formatter are built on XmlReader. To read JSON, however, I think it's possible for someone to implement ISyndicationFeedReader and ISyndicationFeedFormatter based on a "JsonReader". It does not have to be implemented by the "framework", but it will good for the framework to provide proper foundation.

What is the model in handling namespaced extensions?

This is an interesting question. Current design probably needs more work for this. @dodyg do you have any more specific example in mind? @drago-draganov do you have any thoughts about this?

dodyg commented 7 years ago

http://scripting.com/rss.xml uses a source:outline extension on RSS
https://en.wikipedia.org/wiki/Media_RSS (https://www.youtube.com/feeds/videos.xml?channel_id=UCBcRF18a7Qf58cCRy5xuWwQ)
https://sports.yahoo.com/mlb/teams/bos/rss.xml uses Dublin Core extension

I bet there are plenty more used internally.

zhenlan commented 7 years ago

@dodyg thanks for the examples. We will look into them.

dodyg commented 7 years ago

I hope it can be supported as the extension mechanism for RSS and ATOM is the same. For enterprise development I have seen people writing a custom RSS parser just to process extensions.

More:

SharePoint RSS Extension

lviana1 commented 7 years ago

You mention there will be default parsers for RSS and Atom feeds, but you don't specify which versions. There are multiple versions in use out there, with the important ones being: RSS 1.0 (RDF), RSS 2.0, Atom 0.3, and Atom 1.0. Ideally there would be support for all of these. Support for JSON Feed is also a good idea, IMO.

As for namespace extensions, here's a directory of different ones: http://www.feedforall.com/directory-namespace.htm. The big ones are really: iTunes, Media RSS, Dublin Core, and Content. Ideally there would be support for these as well.

I'm glad to hear that there will be a more forgiven date parser. There are many feeds out there with nonconforming dates and other fields. A less strict parser that allows the raw data to be extracted would make this library significantly more useful.

zhenlan commented 7 years ago

@lviana1 thanks for the question and additional information. Our plan is to support RSS 2.0 and Atom 1.0.

I am excited to announce that we just released a preview of Microsoft.SyndicationFeed on nuget.org. Here is a bit instructions and examples to get you started. We are eager to hear your feedback. Please let us know how it works for you.

RehanSaeed commented 7 years ago

An Atom feed writer does not currently exist in the new API. Someone has raised https://github.com/dotnet/wcf/issues/2169 to address this issue. Any guestimates for when one might be built?