JSON Feed support?

[ksuther]

Would adding JSON Feed support to Sparkle be useful to anyone?

There's not any specific advantage for Sparkle itself, but it would make creating, managing, and reading appcasts a whole lot easier elsewhere. For example, tasks such as writing a script to automatically build an appcast, reading an appcast to ensure a download link is always downloading the latest version, and often be more straightforward with JSON than XML.

[dimitribouniol]

A specific use case could be to generate personalized appcasts much more easily via an API.

[jkbullard]

This has come up repeatedly. See #427 and #905.

Since #905, Sparkle has added an appcast-generating script.

[kornelski]

If JSON Feed catches on, we might follow.

So far I've been reluctant to add JSON support, because:

• It'd fragment the ecosystem. 3rd party tools consuming feeds would have to add JSON-sparkle-feed support as well.
• We can't drop RSS, so we'd have to maintain two formats.
• We now have the generate_appcast tool, so there's a way to avoid writing XML.
• There are some things that are easier in JSON, but it's not clearly superior. The syntax handling is also draconian, and inline HTML is harder to write.
• It doesn't solve any of our problems. If we were to change the feeds, addition of signatures (#971) seems like a higher priority, but because XML supports comments, it's easier to do them for XML than JSON.

In my experience with writing generate_appcast, dealing with XML wasn't too bad. Overall it was maybe 10% of the effort, and mainly because it tries to update a feed in-place, preserve unknown items and pretty-print.

Dealing with archives, selecting versions, generating deltas, and signatures took most of the work, and none of it is any easier in JSON. And if you just want to customize a feed a little, you can use any of the countless HTML templating tools.

[bdkjones]

An old thread, but I just googled this again to see if Sparkle had made progress on this front and discovered my issue (#905) from three years ago. I understand all the points above. But in my case, I'm now using my appcast file to transmit other information about updates that I use in some custom logic. To do that, I need to parse the file.

Here's how that would go with JSON:

1. Ask NSJSONSerialization to parse the file into an array of dictionaries, each representing an available update.
2. Use the data.

Here's how it goes with XML:

1. Set up NSXMLParser.
2. Set up a delegate for NSXMLParser.
3. Start parsing, receiving a delegate callback for every damn thing in the file.
4. Write a bunch of fragile, custom code to determine when the parser has hit information I'm interested in. Also keep track of which "update" we're talking about. Hell, we better just create an entire model class for this nonsense.
5. Waste a ton of time assembling all this stuff that's BASICALLY just an array of dictionaries, but NSXMLParser can't be told that so we have to do it all by hand.

In short: yes, JSON has limitations. But the tools to parse it are more modern, faster, and better than the tools to parse XML (at least in Foundation). Anyone who wants to do custom things with Sparkle's Appcast would be better off with JSON.

And yes, I realize I could just create a separate file. But then I have two files to maintain when I release an update. My goal is to have a single source of truth that's also modern and flexible; not tied to the 1990's.

[kornelski]

As for the original issue:

• it looks like JSON Feed didn't go anywhere.

• RSS isn't doing great either, so maybe it'd be OK to give up on compatibility.

• Still, Sparkle can't just drop XML, because that would be disruptive to existing users, so at best it has to support both, probably forever.

• We're now moving from Sparkle 1.x to 2.x, so fragmenting that further may not be a great idea. But maybe adding JSON to 2.x only could give users a reason to upgrade :)

Sadly Sparkle has bet on the wrong horse (that's from google trends with the date first Sparkle commit marked)

[kornelski]

As for the metadata - could Sparkle help you with that metadata?

It internally parses the feed into a dictionary, so maybe it could expose that, including some custom elements or attributes. You could even put JSON in an XML attribute if you want :)

[bdkjones]

It seems like JSON support wouldn’t be all that monumental of a task. Sparkle already uses a custom data class to hold information about updates. Populating that from a JSON file should be quite a bit easier than populating it from RSS/XML.

The third parties that implement Sparkle tracking (MacUpdate, etc.) would probably LOVE to use JSON instead because it’s also easier to parse in JavaScript and server-side languages.

Next, the XML stuff is done and tested. XML itself isn’t evolving, so there’s nothing Sparkle will need to do to keep supporting it. If new capabilities are added, those need only be added to the JSON side of things because (A) any new capabilities would be sugar—the core purpose of Sparkle is met with the existing XML feeds and (B) it would encourage users to move to JSON.

As for comments and html: are those frequently used in Appcasts? I would guess most folks handle release notes with an external link.

[zorgiepoo]

Signing appcasts is a higher priority from my point of view before considering the worthiness of this feature. JSON lacking comments makes that harder to do.

Embedded release notes which are used by lots of developers are inferior in JSON. Lacking of comments makes documentation and debugging appcasts for me harder. Every supported feature needs to be spec'd out to ensure soundness including namespaces (so a developer's own elements don't conflict with future ones), xml:lang / localization to any (or select few) elements, how future extensibility of existing elements will work, etc.

I don't think the XML parser code in generate_appcast is as complicated as an earlier comment here describes.

This is a big task on documentation and generate_appcast front.

[bdkjones]

{
"Comment": "Why does everyone keep saying JSON can't do comments?"
}

[zorgiepoo]

That doesn't help with appcast signing and isn't natural (which is important for documentation).

[sindresorhus]

Is this issue accepted?

[bdkjones]

@sindresorhus I think the general consensus is that Sparkle is not opposed to supporting JSON feeds, but it would have to be in addition to XML and nobody wants to do the work.

There was some more direct opposition in #905 by @kornelski and others, but that was back in 2016. In the ensuing 7 years, I think JSON has become even more ascendent, so direct opposition may have waned.

I, for one, would still love to ditch XML for JSON.

[zorgiepoo]

A first step is creating/showing a rough spec/samples and mapping out all of Sparkle's existing features (1 2) including support for localizations, embedded release notes, and the value that namespaces is providing us. We can ignore feed signing for now (previous ideas leveraged off of a preceding XML comment distinct from the rest of the document). If that is sound, then we can discuss if it's a good idea and if it should be accepted (maintenance vs cost of two formats, etc; It seems clear for example JSON will be inferior for embedded release notes, but this needs to at least be possible.)

Then of course is people volunteering to do the initial work needed to be done in the framework, adding JSON tests, updating the documentation to support two formats, and updating generate_appcast to support JSON which still deals with "hard" tasks unrelated to the serialization format.

The resistance to JSON before is basically because while it's nicer to write/parse, it doesn't add any useful feature we want to use. In fact, we basically lose features we leverage like comments and the ease of use of inline release notes. XML for us is already a sunk cost we have to live with, and supporting JSON may imply more maintenance costs in the future past the initial PRs.

[bdkjones]

@zorgiepoo An alternative to consider: add JSON support for those who don't need all the heavy-hitting stuff in the XML format. Since JSON isn't replacing XML, there's no need for it to have all the same capabilities. Developers who need to embed release notes or localize can use XML. Developers who just want a feed that lists versions and URLs can adopt JSON.

Different formats for different scenarios. There's not really a reason they need to be identical.

[zorgiepoo]

I don't want to add JSON if we won't provide as full/good support for XML as we do today. Maybe one day JSON could become what we primarily recommend (but it may take a long time and not happen initially).

[bdkjones]

Somewhere, some bank software engineer is saying the same thing about COBOL.

@sindresorhus it appears we'll have to try again in 2030. See you then.

[mredig]

I'll beat a dead horse and add +1 to desiring JSON support. :)