No JSON based social syntax

[melvincarvalho]

The WG charter states:

A JSON-based syntax to allow the transfer of social information

Yet I was unable to locate any JSON based examples in the text (tho perhaps I missed something)

Could this be made available for review?

[akuckartz]

JSON-LD please.

EDIT: If JSON is used then it should be JSON-LD.

[sandhawke]

Hmm. This is going to require some discussion.

Charter issues aside, can you provide a use case where json (or json-ld) would be overwhelmingly better than form encoding? I say "overwhelmingly", because I think it's clear from current adoption that form encoding works well for this, and also that having only one mandated syntax works well.

From what I've heard, expressing webmentions in RDF or json is easy enough when desired, but that doesn't mean it needs to be what's transmitted.

(Again, ignoring the charter question for the moment.)

[aaronpk]

The Webmention spec itself does not claim to implement the "social syntax" part of the charter. Webmention can be used with JSON documents, as is described in 3.2.2 Verification. Additionally, there has already been an implementation using Webmention with JSON-LD, which is thoroughly documented at http://lombardpress.org/2016/04/16/iiif-webmentions/

[melvincarvalho]

It's kind of hard to leave the charter aside. Because I think that it was hard to come to consensus on anything in the group, but the one thing we did come to consensus on was that we'd like to be using JSON to be passing things around.

So the main motivation for this is interoperability. The social web benefits from more participants to grow the network effect. I think this is especially true in this group where the large W3C members have mostly not joined meaning we are largely a grass roots effort working with organic growth rather than widespread adoption (id be happy to be corrected, if that's not accurate).

So the case for interoperability means that we stand a stronger chance by working together with a common ground. The issue with form variables is that the namespaces are stripped off and need to be obtained out of band (by reading the spec) and then hard coded. That's going to make interop with systems like Solid (or anything following W3C RECs) that much harder.

Why not have two examples implementors can use, one in JSON and one with form encoded variables?

[aaronpk]

What I mean is that Webmention does not (and should not) define the social syntax. That's why we have Activity Streams 2.0 for example. We've already seen that Webmention can be used by JSON-based systems (JSON-LD even), so I don't see the problem here. If more implementations start using Webmention with JSON documents, then it may warrant adding some explicit examples to the spec.

[aaronpk]

There was some previous discussion within the group about using JSON for the actual Webmention payload itself, but we ended up resolving not to go down that path: https://github.com/aaronpk/webmention/issues/4

[kevinmarks]

The charter also says "Each of these technologies should not be tightly-coupled but can allow general purpose use." Melvin is proposing tight coupling here.

[sandhawke]

@melvincarvalho I'm not asking you to forget about the charter, I'm asking you to keep the two issues separate. Would it help to give them different issue numbers?

On the non-charter issue, you start to make a case, but I don't personally find it compelling, and I'm a huge fan of JSON and/or RDF. I don't see where there's a big payoff in Webmention using JSON as its payload. Solid, for example, seems to have decided not to use Webmention at all, not because of the lack of RDF payload, but because posts are authenticated anyway so the payload is encouraged to be more than a mention.

On the charter issue, at W3C, interpretation of the charter is up to group chair(s). I just met with the chairs (@tantek @evanp @lehors) and they agreed with the position @aaronpk stated above, that it's the social syntax (that is, AS2) that needs to be in JSON, not that every protocol has to use JSON payloads for everything. So Webmention using form-encoding instead of JSON does not violate the charter.

[melvincarvalho]

@sandhawke being not a big fan of RDF, JSON (or even of solid) makes it slightly challenging for me to feel encouraged that we can have an unbiased and productive conversation, but we can give it a try!

Re: splitting the issue into 2, we could do that, if you feel it's a good idea. I'll try and keep the points separate then.

In terms of the charter, thanks for talking to the chairs, Im not sure I agree that this interpretation represents consensus of the group. I would be interested to understand the position of other members of the WG / W3C.

Re: interoperability, I felt I gave a compelling reason, namely the network effect. Dont you think that multiple communities working with a common ground have a better chance of success?

I dont see how solid could even theoretically use webmention when it uses syntactic sugar instead of namespaces, and the namespaces are communicated out of bound, in the spec. I think the point here is that using existing web standards and W3C RECs, as Solid does, greatly increases interop.

Introducing a new system is just a pain for hetrogeneous systems that might have wanted to cooperate but now have an unreasonably high bar. Isnt the point of W3C RECs so that, if you follow them you'll be able to interop with other systems much more easily. This work seems to be balkanizing, rather than, bringing systems together. Worse still, those that are following existing w3c recommendations (which is a lot of work) are seemingly penalized!

[melvincarvalho]

Let me perhaps ask a slightly different way. Is there already an example of a W3C REC where form encoded variables are passed around? I would find it instructive to review, if so.

[sandhawke]

I'm not sure how you read, "I'm a huge fan of JSON and/or RDF" and concluded I wasn't one.

I'm suspect if you search the minutes of the thousands of Working Group meetings I've participated in, you could also find me saying I'm not a fan of RDF. That's kind of a love/hate relationship. But I'm definitely a fan of JSON.

[sandhawke]

I have no idea what W3C REC use form-encoding aside from HTML. I think HTML is enough.

" I would be interested to understand the position of other members of the WG / W3C."

That's not how interpreting charters works. If you left interpretation of the charter to the group, then the group could re-interpret its charter at will.

I dont see how solid could even theoretically use webmention when it uses syntactic sugar instead of namespaces, and the namespaces are communicated out of bound, in the spec.

Well, back before Webmention went to CR, when it could have potentially been changed, the solid folks discussed it and decided they weren't interested in pursuing aligning with Webmention.

It could be done in a backward compatible way -- and still could -- by defining a form-encoding-to-RDF mapping, basically borrowing JSON-LD. And like JSON-LD, where the context MAY be specified by the protocol context, one could define a default @context for the webmention protocol.

One could do all of that, but frankly Webmention is so easy to implement, there's just no motivation, as far as I can see.

[melvincarvalho]

I'm not sure how you read, "I'm a huge fan of JSON and/or RDF" and concluded I wasn't one.

I'm suspect if you search the minutes of the thousands of Working Group meetings I've participated in, you could also find me saying I'm not a fan of RDF. That's kind of a love/hate relationship. But I'm definitely a fan of JSON.

@sandhawke Im really sorry for my misreading of this. It's late here and I've been multi tasking. Total apologies. I guess we've had one on one conversations about RDF in the past, and that put the idea in my head. Not an excuse, sorry anyway. I hope my following comments, could be taken at face value.

[melvincarvalho]

It could be done in a backward compatible way -- and still could -- by defining a form-encoding-to-RDF mapping, basically borrowing JSON-LD. And like JSON-LD, where the context MAY be specified by the protocol context, one could define a default @context for the webmention protocol.

Nice idea. But I wonder if this would actually work? Because JSON-LD distinguishes between literals and URIs eg using JSON objects. Is the same possible with form encoded URIs.

At this point I feel that using an existing REC such as JSON-LD, Turtle, RDF/XML etc. is a superior choice. I dont think Solid (and other implementations) should forced to change to support form encoded variables, in order to interop. Surely if the argument is that webmention is easy to implement, it's also easy to implement in JSON too, and its incumbent on the system to justify why web standards are not being reused, when this could easily happen, and allow an increased network effect?

[melvincarvalho]

Well, back before Webmention went to CR, when it could have potentially been changed, the solid folks discussed it and decided they weren't interested in pursuing aligning with Webmention.

FWIW I did raise this a couple of times I think on the mailing list in July of last year.

[sandhawke]

Form encoding is clearly a standard. It's not as flexible as JSON, but Webmention doesn't need that flexibility.

One can't just add JSON to Webmention without making dozens of implementors change their code if they want to stay in compliance. And you haven't actually said anything that would be easier for anyone if they did.

[melvincarvalho]

So Im saying to make those small changes, which in the vast majority of cases affect only one user, to resuse an existing W3C REC, rather than create a new system.

I think we're possibly going round in circles when I say webmention is breaking interop with standards compliance systems (eg solid), and you saying that standards compliant systems dont want to interop.

If form encoded variables are going to become part of a W3C REC, it's something new (at least to me), and it seems to me we should solve the problem in a generic way ie in a way that is universal, extensible, interoperable etc. If it's not something new, I'd love to be pointed to where it's been done before, that'd be a great precedent.

I've not yet seen a good argument for not using standards, for this use case. I suppose that widespread adoption could be one such reason. But I think adoption is small, perhaps less than 0.01% of the social web? I guess Im asking why is the this should be a W3C recommendation and not a note, when it breaks interop with existing standards.

[kevinmarks]

On 24 May 2016 11:40 pm, "Melvin Carvalho" notifications@github.com wrote:

So Im saying to make those small changes, which in the vast majority of cases affect only one user, to resuse an existing W3C REC, rather than create a new system.

I think we're possibly going round in circles when I say webmention is breaking interop with standards compliance systems (eg solid), and you saying that standards compliant systems dont want to be interop.

If form encoded variables are going to become part of a W3C REC, it's something new (at least to me), and it seems to me we should solve the problem in a generic way ie in a way that is universal, extensible, interoperable etc. If it's not something new, I'd love to be pointed to where it's been done before, that'd be a great precedent.

well, it started in 1993 in HTML+:

https://www.w3.org/MarkUp/HTMLPlus/htmlplus_42.html

It was in HTML3

https://www.w3.org/MarkUp/html3/forms.html

HTML4

https://www.w3.org/TR/html4/interact/forms.html

and HTML5

https://www.w3.org/TR/html5/forms.html

It's been the dominant form of client server communication for around 20 years, It's implemented by browsers and servers on around 4 billion computers.

So you can keep pretending it's some new invention, and we can close this issue

I've not yet seen a good argument for not using standards, for this use case. I suppose that widespread adoption could be one such reason. But I think adoption is small, perhaps less than 0.01% of the social web? I guess Im asking what is the this should be a W3C recommendation and not a note, when it breaks interop with existing standards.

You clearly have a very personal definition of the meaning of Standards.

[melvincarvalho]

@kevinmarks It's best to avoid words like 'pretend' when talking about other people's actions and intentions. Good typically doesn't come of it...

[kevinmarks]

I'm using "pretend" in the sense of "make a claim". Are you using "standards" in the sense of "flags that are personal emblems to identify participants in a battle"? On 25 May 2016 12:40 am, "Melvin Carvalho" notifications@github.com wrote:

@kevinmarks https://github.com/kevinmarks It's best to avoid words like 'pretend' when talking about other people's actions and intentions. Good typically doesn't come of it...

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/aaronpk/webmention/issues/39#issuecomment-221496855

[sandhawke]

@melvincarvalho , your argument that somehow form-encoding isn't "standard" or Webmention isn't "standards compliant" really feels quite absurd to me.

I don't know exactly what your proposed "small change" is. I'm guessing it's either (1) use JSON-LD instead of form-encoding, or (2) require Webmention receivers to understand both form-encoding and JSON-LD.

In either case, that's going to be a pain for all current implementors. Option 2 would be a pain for all future implementors. It's completely unclear how it would actually make anyone's life better, while it's clear making the change would be a pain for all the people who are currently using the technology. Probably, they simply wouldn't make the change, and you'd have a dead fork of the spec.

I mean, how would convince a current implementor, in a currently working ecosystem, that they're better off breaking everything and switching to a new protocol that differs only in syntactic sugar? Handwaving about "standards compliance" certainly wont do it.

[melvincarvalho]

Im not quite sure I know how to respond to this. Let me ask first off, do you believe, in creating standards we should use URIs to name things?

[melvincarvalho]

I think, in general, I am perhaps misunderstanding the difference between REC and Note, and I would be very happy to understand that better, or be pointed to something that can explain it better. Im really quite happy for the webmenion ecosytem to be a note, but my, perhaps naive, understand of REC implies "this is how we should solve this kind of problem", which I am slightly more cautious about. I would be happy to understand this better.

[sandhawke]

I'm not quite sure I know how to respond to this. Let me ask first off, do you believe, in creating standards we should use URIs to name things?

I believe using URIs to name things can be a useful tool in the effort to achieve decentralized extensibility. The last 15+ years of experience with xmlns and xhtml and rdf has convinced me the practice is a mixed blessing and should be used with care. In the current design of webmention, I don't see URIs for property names as justified.

It would be nice if webmention were more extensible, but URIs for property names would be little help in that, since the sender wouldn't know what vocabulary the receiver would be capable of handling. Just sending stuff blindly to an endpoint seems less-than-useful. If it's not blind -- if there's a well known set of options (eg vouch) -- then we don't need URIs for it.

[sandhawke]

I think, in general, I am perhaps misunderstanding the difference between REC and Note, and I would be very happy to understand that better, or be pointed to something that can explain it better. Im really quite happy for the webmenion ecosytem to be a note, but my, perhaps naive, understand of REC implies "this is how we should solve this kind of problem", which I am slightly more cautious about. I would be happy to understand this better.

The difference is that with a Recommendation, a critical mass of the member organizations thought it was worthwhile, a Working Group thought it was work making a Recommendation, and the various Rec Track steps (like demonstrating interoperability) were followed. That's commonly understood, I think, as meaning that Recommendations are more carefully thought out and more people are interested in them.

Webmention makes more sense as a Recommendation than a Note because people do seem eager to implement it, and folks are (it appears at present) happy to carry it through the Rec Track.

Notes are for when there's just not enough interest, or (not relevant here) it often makes sense if they don't have normative content.

[melvincarvalho]

@sandhawke thanks for explaining, that is very helpful!

So it seems to me that webmention can be solved in one of two ways. One is by reusing existing W3C RECs such as JSON-LD, and the other is a bespoke method using form encoded variables, as is currently documented in the spec, and has a small implementation base, with a relatively large representation in this WG, that's not a criticism (all great movements start out small!), just trying to reflect the actual position.

Why should we not, at a minimum, show an existing standards compliant method in the spec text? e.g. An example of how to do it with form encoded variables and an example using JSON.

Form encoded version:

source=https://waterpigs.example/post-by-barnaby&
target=https://aaronpk.example/post-by-aaron

JSON-LD version:

{
  "@context": "http://www.w3.org/ns/webmention",
  "source": "https://waterpigs.example/post-by-barnaby",
  "target": "https://aaronpk.example/post-by-aaron"
}

To the extent that webmention is "just a signaling protocol" that says to a server, "hey, look at this part of the web which you may not have seen before" you could imagine it working with a simple HTTP GET and query string parameters. Typically query string parameters would not expect namespaces or contexts (but maybe we have become conditioned to think this). But it seems a bit more than that, in that it can be extended it seems to more general cases.

[sandhawke]

That's my option 2. It makes implementating each receiver harder, because now it has to understand two syntaxes. And what's the real benefit? Can you express that benefit in some way all the current implementors will find compelling? What can you say to them that will make them want to go back to their code and add json support? And in your explanation don't make an appeal to authority, explain how making this change will benefit them personally or benefit humanity. It's a high bar.

Also, don't say the community is small now compared to where it will be; in my experience, that's something that only matters in theory, not practice. Abandoning your early adopters before establishing a new user base means you have nobody. If you want to establish a new user base, go right ahead, but then one needs to call it something other than webmention to avoid confusion. (It sounds like you want to dust off Semantic Pingback. Or just use Solid Inbox.)

[melvincarvalho]

I think we're not really communicating. Im trying to express the benefit of interop leading to growth in the network effect, and you dont find it compelling. I guess about all I can say is that I respect your opinion, but mine differs.

[rhiaro]

FWIW I wrote up a draft spec to bridge between webmention and LDP implementations.

https://github.com/rhiaro/wm-ldp/blob/master/index.md

The first half is what to do if you have a webmention implementation and want to talk to plain LDP servers. The second half (more relevant for you @melvincarvalho) is what to do if you have an LDP server or a Solid application and want to receive from / send to plain webmention implementations.

Both ends seem pretty straightforward to me, this isn't a huge gap to plug.

[akuckartz]

@melvincarvalho If I understand the comments here correctly then people seem to be missing an explanation how the proposed change would "lead to growth in the network effect". I think that this is a legitimate question.

[melvincarvalho]

@akuckartz I would have thought that to be self evident. Millions of sites are already using JSON-LD even though its only a couple of years old. Large companies such as Google have said they like it. Solid and other systems are using it. And it's growing. All these factors contribute to a network effect.

[akuckartz]

@melvincarvalho You do not need to convince me that it would make sense to use LD everywhere. The task I see in this issue is to convince others that the proposed change to the webmention specification has a significant positive effect and that there are good reasons to assume that the size of that effect is larger than the effect due to the resulting cost of implementing the change.

(I have not had a really close look at webmention myself and therefore will not attempt to analyse that currently.)

[melvincarvalho]

The implementation change is tiny. As a programmer I can say that. In fact you'll probably get more tooling for free making the cost possibly even negative!

[akuckartz]

Question: What would be the effect of replacing the form encoded version by the JSON-LD version in the spec ?

[melvincarvalho]

@akuckartz it's a good idea. If someone has cycles to write it. It would be a great guide for someone that wished to implement it that way. Unfortunately I just have too many other projects I have to take care of at this time, or id volunteer.

[melvincarvalho]

@rhiaro do you think you might want to have a go at the above, if you have time free? ^^ I hope im not imposing on you by asking :)

[aaronpk]

I'm just going to point back to what @sandhawke said:

Abandoning your early adopters before establishing a new user base means you have nobody.

The time to suggest a 100% breaking change to the spec has long passed. There is already a lot of written and deployed code that sends webmentions using form-encoded syntax. It's not a matter of simply making a "tiny" change to switch to JSON, since it will affect senders as well as receivers, some of which are using libraries on both sides, not to mention the fact that this software is deployed quite widely and in places where a single person can't even mandate that it be updated. Such is the situation when a spec gets widely implemented.

[akuckartz]

@aaronpk I understand your view but that argument does not convince me. If the change of an unfinished specification would require a rather small change in the code of current implementations than I would not agree that it is a "100% breaking change".

[kevinmarks]

@rhiaro's document https://github.com/rhiaro/wm-ldp/blob/master/index.md is a very useful contribution here.

I'm happy to stipulate that many people find RDF useful to represent things, and a way to map between its worldview and webmention is both helpful for them and useful for interop. That does not mean that it should be mandated for everyone else to parse, any more than representations in MySQL, C++ classes or Elixir Records should be.

Form encoding is the time-tested boundary between implementation structures that minimizes the exposed surface area. If you want to do something else, you are free to do so, but changing this spec will not change other people's code. This is documentation, not legislation. Saying "The implementation change is tiny." and "Unfortunately I just have too many other projects I have to take care of at this time, or id volunteer." does not make the case. 🤦

[tantek]

@sandhawke this issue is rapidly going off-topic / growing beyond original scope / going in circles.

Per your comment in https://github.com/aaronpk/webmention/issues/39#issuecomment-221340025 there's no charter issue, and thus this issue should be closed.

If there are implementers that want to brainstorm extensions to webmention, whether JSON(LD) or otherwise, I encourage them to open new specific issues for those new feature requests (citing their existing implementations would be useful), which can be considered for a future version per the point you made "Well, back before Webmention went to CR, when it could have potentially been changed".

[akuckartz]

@tantek It does not make sense to create a new issue when the current one can be used.

[aaronpk]

This issue is called "No JSON based social syntax", which is totally unrelated to the request to convert the webmention POST request to JSON format.

[akuckartz]

@aaronpk Sorry, but that is a bit like hair-splitting. This issue can be retitled appropriately.

[tantek]

@akuckartz morphing issues is considered bad practice in issue tracking systems. This started (primarily) as a charter issue, that's been resolved. It is customary practice to encourage creation of new issue(s) for anything ancillary discovered as part of the discussion. In particular if you're interested in the specifics of how W3C uses / should use Github, I encourage you to join discussions elsewhere for that: https://www.w3.org/wiki/GitHub

[akuckartz]

I agree that it is very bad that this issue was raised so late (although I have heard vocal concerns a long time ego).

But the core of this issue from the beginning was about the fact that JSON(-LD) is not supported/used by the current version of the webmention specification. That certainly was not "ancillary discovered as part of the discussion".

@tantek Thanks for the link, but I did not find anything relevant for the current issue there.

[sandhawke]

@akuckartz wrote

If the change of an unfinished specification would require a rather small change in the code of current implementations than I would not agree that it is a "100% breaking change".

I think a reasonable definition of "100% breaking change" (for a spec) is: a change which makes 100% of the implementations no longer conform to the spec. Switching the webmention syntax from form-encoding to JSON-LD would therefore be a 100% breaking change. Every single implementation would have to be modified (and all at exactly the same time). Mandating that receivers accept JSON-LD would be 100% breaking for receivers (unless there are some that currently accept JSON-LD? Maybe @rhiaro's does), which would probably be something like 33% breaking, assuming there are twice as many receiver implementations as sender implementations.

Rather than split off another issue, I suggest we close this one. @melvincarvalho @akuckartz do you want to keep talking about this? If so, can you raise another issue, specifically on the proposal that mentions payload syntax be JSON-LD instead of Form-Encoding, or in addition to Form-Encoding? (I suggest you pick one. I'm pretty sure I've already explained why neither one is tenable, but we can keep talking about it.)

[aaronpk]

Based on the requests from a chair and w3c team contact, I am closing this issue. @melvincarvalho @akuckartz please see the suggestions above for ways to pick up this discussion again if you feel it is necessary. In the future, please keep the discussions in these issues focused and on the original topic.

[akuckartz]

For the record: I strongly object to the claim that the contributions were not focused on the original topic.

[melvincarvalho]

Also strongly object.

I agree that it is very bad that this issue was raised so late (although have heard vocal concerns a long time ego).

I raised this in July of 2015 on the mailing list.

[melvincarvalho]

FWIW one of the main reasons I stopped following this closely was that both @aaronpk and @tantek were closing issues prematurely in a heavy handed way. At one point I was on the admin group of github and I was removed unilateral, without explanation. While issues were closed down and several members of the WG wanted to discuss further. This process is a joke. All I can say is that I object in the strongest possibly way.