LDO "get" method is too ambiguous

[jdesrosiers]

I'm just getting around to reviewing the new Hyper-Schema spec and I have a few issues. Sorry, it's a bit late. I'm doing the best I can to keep up with the lightning fast pace this project suddenly started moving at.

If "method" is "get", the link identifies how a user can compute the URI of an arbitrary resource. For example, how compute a link to a page of search results relating to the instance, for a user-selected query term. Despite being named after GET, there is no constraint on the method or protocol used to interact with the remote resource.

The problem I have is the ambiguity. Just like HTML is unambiguous enough for a web browser to generate a UI from HTML, a Hyper-Schema client should be capable of generating a UI (graphical or otherwise) from a hyper-json (JSON Document + Hyper-Schema). Based on this wording, it seems like you can do almost anything you want. What is a generic client supposed to generate when it encounters this LDO?

This is a preserving-the-spirit-of-hyper-schema level issue for me. If I'm missing something, please explain and lets put a better explanation into the next draft of the spec. If I haven't misunderstood, this change is a deal-breaker. I can't use Hyper-Schema like this without assigning semantics to things that aren't in the spec.

[jdesrosiers]

Same thing goes for "post", although it's not quite as bad. The only time you send a request body to the server and it is ambiguous what you should be sending, is when you are doing a POST operation. So, "post" should imply a POST operation. But, there are too many hoops you have to jump through to get to that conclusion.

If "method" is "post", the link specifies how a user can construct a document to submit to the link target for evaluation.

[handrews]

I just attempted to do something about this and went down in flames (the issues I opened are already closed and are not recommended reading). I appreciate @awwright 's patience although I still share a lot of your concerns, @jdesrosiers .

My current understanding is this:

• All of this terminology is by analogy with HTML forms, and therefore incorporates how the HTML form element behavior has evolved as the web has expanded beyond HTTP/1.0, even though HTML5 still has pretty much the same form syntax.
• A link that takes input is a form. There's nothing more to it than that.
• A "get" link/form encodes its input in the URI. That's all that "get" means. It has nothing to do with HTTP GET or any sort of more abstracted retrieval operation. It does not mean that sending input to the form is safe.
• A "post" link/form encodes its input in the request body. That's all that "post" means. It has nothing to do with whether sending input to the form involves safe, unsafe, idempotent, or non-idempotent behavior, nor is it related to the protocol method used.

---

I'm more or less fine with the idea that there only links that do or don't take input. I'm still working through the details, but this is why I like JSON Home as a complement to JSON Hyper-Schema, because it is oriented towards hinting at the semantics and mechanisms that Hyper-Schema is (apparently) intentionally avoiding.

I somewhat dislike calling links that take input "forms". I am particularly confused that "form" is considered to be good terminology here, but the "create-form" and "edit-form" links are considered invalid to use. This means that we're using "form" for different things, and I do not really understand why. The link rels, I'm told, are only for interactive forms, but I can't figure out how the RFC for them imposes any such constraint.

I very strongly dislike both the use of the terms "get" and "post" and their use to separate encoding of input into the URI vs the request body.

You've already observed the horribly misleading nature of these terms given the specified usage. So have I, including an extensive example of why I think it is a problem for adopting Hyper-Schema because it results in very poor UX for schema designers (I can elaborate on this concept of UX if needed).

@awwright 's response was:

We're designing a specification whose audience is developers of validation libraries. So when I say that "get" and "post" are opaque tokens, that's because we can make the vocabulary more intuitive later, right now we're more concerned with how is this going to work.

Needles to say, I disagree with this- the time to fix vocabulary is now, before too many people adopt (and likely misuse compared to the current specification) the current vocabulary.

As far as encoding:

• We have far more powerful tools available with URI templates
• It should be possible to resolve data from both the instance and user input whether it ends up in the URI or a request body

I'm not sure why we're preserving this split explicitly when it is rather specific to HTML's usage of HTTP/1.0

[handrews]

I forgot to elaborate on the "how the HTML form element behavior has evolved" comment. In the epic "does anyone actually use hyperschema?" issue, @awwright said:

Even in HTML, method="get" and method="post" doesn't necessarily mean an HTTP operation using that method. Indeed, <form method="post" action="mailto:robot@example.com"> ... is supposed to send an email, not make an HTTP request.

... and I've been trying to work with that since. But I feel a bit better about my confusion, because W3C's HTML5 specification still defines method as:

method - HTTP method to use for form submission

You have to dig down into the algorithm for form-submission to find this part:

Otherwise, select the appropriate row in the table below based on the value of scheme as given by the first cell of each row. Then, select the appropriate cell on that row based on the value of method as given in the first cell of each column. Then, jump to the steps named in that cell and defined below the table.

            GET                 POST
http        Mutate action URL   Submit as entity body
https       Mutate action URL   Submit as entity body
ftp         Get action URL      Get action URL
javascript  Get action URL      Get action URL
data        Get action URL      Post to data:
mailto      Mail with headers   Mail as body

If scheme is not one of those listed in this table, then the behavior is not defined by this specification. User agents should, in the absence of another specification defining this, act in a manner analogous to that defined in this specification for similar schemes.

[... much more detail about the stuff in the table ...]

It's not clear to me why we should model ourselves after something that's basically a switch statement on protocol. As far as I can tell HTML having something like this is an artifact of it originally being designed for HTTP/1.0 and needing to map that design into other protocols. We can make things a lot better by designing something that is not constrained by HTTP/1.0 history.

Some of these have no distinct "post" behavior. HTTP and email have separate actions, but while the "body" concepts are reasonably analogous, email headers and URL query string parameters do not necessarily have the same or even similar semantics.

We should have one way to define what input can be accepted and where to put that input, whether that is through URI templating or message body construction. The "where" should be determined by either mapping the input to URI template variables (in which case it goes in the URI, more flexibly than with form query encoding), or by not mapping it (in which case it goes somewhere else which is probably protocol-dependent as specified by the URI scheme, but not constrained by Hyper-Schema).

That input system should also optionally map instance data into the input (no matter whether the input is for the URI or not). Some input may not be resolvable from the instance, and some URI template variables may not be available for user input, so we should be able to express those things as well.

[handrews]

I just went and looked at draft 04's meta-hyper-schema:

"method": {
    "description": "method for requesting the target of the link (e.g. for HTTP this might be \"GET\" or \"DELETE\")",
    "type": "string"
},

and spec:

This property defines which method can be used to access the target resource. In an HTTP environment, this might be "GET" or "POST" (or other HTTP methods).

Some link relation values imply a set of appropriate HTTP methods to be used for the link. For example, a client might assume that a link with a relation of "edit" can be used in conjuction with the "PUT" HTTP method. If the client does not know which methods might be appropriate, then this SHOULD default to "GET".

So I am not losing my mind, and when I read Hyper-Schema years ago, I did not fail to comprehend how it used this keyword. Specifying full operations by HTTP method was actively encouraged. The things we did at Riverbed followed directly from this definition. It may never have been the right way to think about links, but it was what was advised at the time here.

That means this new wording is actually a major change. (And I feel a lot better about my mental capacity.)

[jdesrosiers]

Thanks, @handrews, that was extremely helpful. We need to do a much better job at communicating this in the spec. We don't want to duplicate the HTMLforms documentation, but right now it makes it sound like the semantics of "method" are largely undefined.

I have not read the HTMLforms spec, but I think what I am hearing is that at least when HTTP(S) is used, "post" not only means "Submit as entity body", but also means the "HTTP method to use for form submission". This interpretation would be consistent with the way browsers implement forms.

So, it is now my understanding that the spec is just saying that when "method" is used, an LDO uses the same semantics as an HTML form. So, this doesn't really change the semantics of "get" and "post" from what they were in draft-04 when using HTTP. @awwright, can you confirm that I understand this correctly?

Am I correct that Hyper-Schema has lost the ability to describe that "PUT" and "DELETE" operations are allowed? Is "method" not allowed to be "PUT" or "DELETE" in an HTML form? I know it works for ajax calls. It would be unacceptable to have to require clients to make an OPTIONS request to know if "DELETE" and "PUT" are allowed. It's just not practical.

If this is correct, we shouldn't have published the new draft with this functionality removed. The "allow" proposal (https://github.com/json-schema-org/json-schema-spec/issues/73), could fix the problem. That or something like it should be added ASAP because Hyper-Schema is incomplete without it.

[handrews]

@jdesrosiers You're welcome! My feeling is that JSON Home, either directly or by the example of its "hints" concept with which "allow" fits, is the answer to the OPTIONS problem. I'm mostly sold on the idea that features that non-authoritatively optimize protocol deficiencies should be separate in some way from the main Hyper-Schema features. And also that it should be possible to construct a fully-functional hypermedia system without them, although it may be slow and inconvenient.

[awwright]

@jdesrosiers I believe you understand correctly. The new draft, for the most part, is reverse-compatible, with some changes in language and, where necessary, definitions in order to bring the language up-to-date with the state of the art of hypermedia systems.

One of those lessons is that links don't tell you how to interact with a resource; they tell you the relationship between resources. Once you know a resource exists, then you select how you want to interact with it. If it's an HTTP resource, you select a method like GET to retrieve the resource, or you use PATCH or PUT if you want to change it, DELETE if you want to remove it, etc. There's no need for the link to re-define everything that's already defined in HTTP.

The HTML spec is increasingly becoming the "Web browser specification" and defines a lot of things that really have nothing to do with HTML (it has a whole section on what to do if the document is not HTML, something I believe grossly out of scope).

So when it says "Get action URL", it's talking about what the behavior of the submit button is, not what the form means. So the behavior between HTML and JSON Schema is still fundamentally the same: A method="get" form tells us how to construct a URI for an arbitrary resource (like a search query), and it is something you can PUT to or PATCH to if you're so inclined.

[awwright]

@handrews Is there some difference between HTTP/1.0 and regular HTTP I'm unaware of? HTTP/1.1 is the latest spec.

Also, I've never had to use OPTIONS, so I guess I'm still unsure exactly what problem it is that you're trying to solve. Most all of the time you can just opportunistically make the request you want to make. If the server doesn't support it, it'll come back and tell you so.

[awwright]

@handrews:

Needles to say, I disagree with this- the time to fix vocabulary is now, before too many people adopt (and likely misuse compared to the current specification) the current vocabulary.

The explicit goal of the latest I-D was to be reverse-compatible, without any updates to the meta-schema. That's all.

If there's a better idea for how to say "this relation accepts user input" or "here's how to encode a POST submission", now's the time to suggest it!

[handrews]

The explicit goal of the latest I-D was to be reverse-compatible, without any updates to the meta-schema. That's all.

If there's a better idea for how to say "this relation accepts user input" or "here's how to encode a POST submission", now's the time to suggest it!

The latest I-D has already been submitted when you wrote that, so you'll have to forgive me for assuming that you were blowing off my concerns in general. Perhaps I should have kept using the "v6" prefix that I dropped because you said you didn't like it?

[handrews]

Is there some difference between HTTP/1.0 and regular HTTP I'm unaware of? HTTP/1.1 is the latest spec.

Actually, HTTP/2 is the latest spec, and transparent use of either HTTP/1.1 or HTTP/2 (and possible some non-HTTP protocols) is a top priority of my current REST project.

HTTP/1.0 only specified GET, POST, and HEAD. It was more of a description of what was implemented at the time rather than a specification for implementation. I don't know whether HTTP/1.0 defining only GET and POST was because of HTML, or HTML only allowing methods of "GET" or "POST" was because of HTTP/1.0. The important point is that HTML5 has never moved away from this, and the messy "get"/"post" terminology is the result.

Were you to start HTML over today with HTTP/2 or even HTTP/1.1 as the primary expected protocol and no legacy constraints, no one would design "method" to take "get" and "post" as its two values. They only even sort-of make sense when you fully understand the history.

[handrews]

So when it says "Get action URL", it's talking about what the behavior of the submit button is, not what the form means. So the behavior between HTML and JSON Schema is still fundamentally the same: A method="get" form tells us how to construct a URI for an arbitrary resource (like a search query), and it is something you can PUT to or PATCH to if you're so inclined.

With all due respect (and based on your work and patience here, that's really quite a lot), you are too close to this to see how poorly it is landing on everyone else who has commented so far. [edit: we're all too close to parts of this- see next comment]

Yes, you've sliced the things implied by the HTML spec that tell the browser how to use the data from the part where the browser actually picks an HTTP method for sending the data, but that is not a separation that is clear from the resulting terminology. To be blunt, even though I understand and support the conceptual model you are pushing with this description, I am not buying the description and terminology in the slightest.

These conversations have significantly helped and improved how I reason about specifying links in hyper-schema. So you are generally convincing on this topic. But when it comes to this way of speaking about things, it's about as unconvincing as anything I've ever encountered. Being technically correct doesn't mean it makes any sense.

(and yes, I'm writing a proposal with alternate terminology and a better coverage of the link/form/instance combinations allowed by HTML but not JSON Hyper-Schema, although with considerable trepidation given your tenacity on this point)

[handrews]

@awwright said:

If there's a better idea for how to say ... "here's how to encode a POST submission"

Why are we even talking about "here's how to encode a POST submission"? You're all about not specifying PUT and PATCH, so we wouldn't talk about 'here's how to encode a PATCH submission". We use Accept-Patch for that.

And please don't tell me you don't mean HTTP POST. You capitalized it, so it reads as an actual HTTP method, and not an HTML form method of "post".

If you did mean an HTML lower-case "post", then clearly you're having trouble keeping this straight which should really go a long way towards explaining why @jdesrosiers and I are pushing back on this as confusing or misleading.

If you actually meant HTTP POST, then I am totally lost and want to hear why we are talking about HTTP POST but shouldn't talk about encoding HTTP PATCH. We should either be talking about both (because POST meant "post" which doesn't mean POST) or neither.

[awwright]

Perhaps I should have kept using the "v6" prefix that I dropped because you said you didn't like it?

Well, I said it's going to be disappearing from the IETF documents. I don't really have a better alternative to suggest, though. The milestones are still draft-5 and draft-6.

[awwright]

Actually, HTTP/2 is the latest spec, and transparent use of either HTTP/1.1 or HTTP/2 (and possible some non-HTTP protocols) is a top priority of my current REST project.

HTTP2 is a completely different encapsulation of all the same messages that can be carried by HTTP/1.1. There's few, if any, semantics changes. The methods available for use in HTTP is defined in the IANA registry, the individual methods are spread out across a number of different specs.

[awwright]

Why are we even talking about "here's how to encode a POST submission"? You're all about not specifying PUT and PATCH, so we wouldn't talk about 'here's how to encode a PATCH submission". We use Accept-Patch for that.

I'm talking about how to construct a document for submission to, and evaluation by, a remote resource. In HTTP, the method that does this is POST, so I'm fine with calling it POSTing for brevity. If instead the remote resource were CoAP, there's an exactly equivalent method.

---

I mean, I guess if you want to get technical, you can do this with GET too. But nobody really knows would even mean. I do think there should be a method for safe submissions of resources, that would be too long to encode into a URI. (Also, you can't associate a media type or other headers with a URI, like you can with POST.)

[handrews]

I'm talking about how to construct a document for submission to, and evaluation by, a remote resource. In HTTP, the method that does this is POST, so I'm fine with calling it POSTing for brevity. If instead the remote resource were CoAP, there's an exactly equivalent method.

@jdesrosiers can you comment on whether you find this switching back and forth between POST and "post" easy to follow? I do not, but I wonder if that is just me and I need to think on it further.

[handrews]

What are we trying to do here?

Let me try approaching it this way. There are three different things that can more-or-less involve links/forms.

[EDIT: I eliminated the term/concept of "resource selection" as @awwright cited it as a reason not to address the substance of the comment.]

1. URI resolution (URI Template resolution or input via "method"="get")
2. Interacting with the linked resource using a representation
3. Sending non-representation* data to the linked resource ("method"="post")

*the non-representation data may actually be a representation of something, but given how the form schema is specified the system isn't explicitly aware of this

URI resolution always happens when using a link (possibly trivially if the href is not a template and there is no "get" input). Sometimes this may be all that happens, because we just want to use the resolved URI as an identifier.

Otherwise, we may interact with the resolved target resource using its representation according to the protocol, or possibly using the source resource's representation depending on the relation type (e.g. the standard "collection" reltype with HTTP suggests that POSTing a representation of this resource class to the target resource will create a resource instance of this class).

If the link is a "post" form, we may interact with the resource by sending it a message including data as specified by the form schema. Depending on the relation type of the form, we may also use the same resolved resource for representation-based interactions.

---

How does HTML do this stuff?

HTML is a presentation media type. Because it's primary concern is presentation, it's mechanism for specifying user input for either the first case (resource selection) or the third case (non-representation data) is identical. HTML doesn't care and doesn't want the form user to need to know what happens when they submit their input. So in HTML, URI resolution based on user input and sending non-representation data from user input to a resource are presented identically. Even though they are very different things underneath.

I think this unification at the presentation layer of two very different technical processes is one thing contributing the difficulty, and a big part of why I do not think HTML is an ideal role model for Hyper-Schema.

Interacting with linked resources through representations is done by using HTML's other linking elements ("a" and "link"). In practice this is just GETting the representation, but the point is that that particularly with "link", that behavior comes from the protocol and relation type and is not directly specified by HTML (I don't remember if HTML allows for any other linking behavior with "a").

HTML forms can be written with pre-populated data, including hidden elements. This is the equivalent of using instance data in hyper-schema. Hidden elements must be supplied as part of the instance, while pre-populated user-editable forms are populated from the instance but can be overridden by user input (this is different from defaulting, which is just about how to handle omitted input).

HTML does not have other forms of URI templating or construction- this is usually done on the server as the HTML is generated, or on the client with JavaScript. I'm going to ignore server code and JavaScript for the rest of this comment.

---

How does JSON Hyper-Schema do stuff?

JSON Hyper-Schema has two separate mechanisms for URI resolution. URI Templates resolve from the instance (the spec allows other sources if and only if the instance does not have the data, and it's not clear that user input would be a valid source even then). The link/form schema with "get" resolves the URI using user input.

It is not possible to override instance data with user input in hyper-schema. This kinda-sorta happens if the URI template has a query string, but the only allowable way to do this is for the user input to entirely replace the query string. And there is no correlation between URI Template variables and the input schema. Since we can't tell what, if anything, we're overriding, it's not really a user override in the way that HTML forms allow.

Interacting with the resource through representations is generally invisible in hyper-schema, aside from whatever might be implied by the relation type (which is from outside of hyper-schema anyway, and equally relevant in HTML).

Submitting non-instance data is handled through the "post" link/form schema. It is not possible in Hyper-Schema to pre-populate the form from the instance at all. An application must manually copy instance data in, and will likely rely on some amount of out-of-band information to figure out how to do that.

---

Are we doing all the stuff we need to do?

To recap, here are the things you can do and what mechanisms exist in static HTML and JSON Hyper-Schema to tell you how to do them.

                                         Static HTML              JSON Hyper-Schema
    URI resolution (from instance)   Prepop hidden "get"**    URI Template
    URI resolution (from user)       "get" form**             "get" form**
    URI resolution (user overrides)  Prepop editable "get"**  (not possible)
    Representation interactions          Element or rel type      Relation type
    Non-repr data (from instance)        Prepop hidden "post"     (not possible)
    Non-repr data (from user)            "post" form              "post" form
    Non-repr data (user overrides)       Prepop editable "post"   (not possible)

\ indicates that the "get" form can only affect the query portion of the URI

It's also worth noting that neither static HTML nor JSON Hyper-Schema can take user input for both URI resolution and non-representation data at the same time. JSON Hyper-Schema can do URI resolultion from the instance alongside non-representation data from the form, but cannot do any other combinations.

Beyond that, here are some really obvious things that jump out about JSON Hyper-Schema:

• It's entirely missing the concept filling out "post" forms from instance data, with or without possible user input overrides
• Its URI resolution from user input mechanism is just as limited as HTML's, despite offering the vastly superior URI Template mechanism for resolving from instances
• It's not possible to use "get" forms and URI templates together to allow overriding instance data with user input

The inability to pre-populate forms from instance data (either at all for "post" or in a meaningful way leveraging URI Templates for "get") is a glaring lack that I've been meaning to file for "post" for quite some time. The "get" scenario is more or less addressed by #52 , although that splits "get" form input entirely away from "post" form input. At this point, I'm not sure that's the right approach anymore.

What about that "get"/"post" split?

Currently, Hyper-Schema follows HTML in "presenting" input forms the same way. But hyper-schema is not a presentation system, and keeping this in its current form also limits us to only being able to use one kind of input at a time. The approach in #52 would allow mixing them, but would require setting each separately as the vars mechanism is part of href and the schema is still entirely separate.

I'm not sure if that separation is a good idea or not.

Arguably, users shouldn't care where their input goes or how it gets there as long as they end up with the correctly identified resource, and should they interact with it using a "post" form, the right data is sent. That would point towards a unified input schema of some sort, that maps the input where it needs to go (and interfaces with URI Templates for instance data, and some analogous approach for getting instance data into the "post" form).

On the other hand, we often want to resolve URIs without dereferencing them, or at least without submitting a "post" form. That would argue for some amount of separation. [EDIT: I'm increasingly coming back around to some sort of separation- notably, Python's "requests" library separates these).

---

What to do?

I think we need to decide what level and nature of separation there should be for user input for URI resolution vs non-representation message construction. When we are using instances based on link relations and/or protocol semantics, we don't need to worry about anything except URI resolution. Once we have the right resource it tells us what we need to know.

It's only when non-representation data comes into the picture that things get complicated.

Is this making any sense to anyone? I think if we sort this out and fill in the functionality gaps, then we might figure out some better terminology for hyper-schema.

[awwright]

@handrews I don't believe "Resource selection" is a concept that makes any sense, so the entire rest of the post is somewhat confusing.

The only thing we have to say about links is that they declare relationships between resources, and there's a variety of things that user agents and users can decide to do with this information. In a Web browser, clicking a link retrieves and visually renders the target resource, for example. Also in a Web browser, stylesheet links specify how to visually render the HTML (data) page. The only place you ever see this behavior is in Web browsers. In other contexts, it might be something you decide to set or delete instead, for example, a physical wall switch in CoAP. CoAP even has methods that HTTP doesn't, like watching, where you receive updates only when the resource changes value.

So to reiterate, links don't restrict behavior, the user (or user agent) determines what they do with a link. It might be different between each type of user agent, it might be different between each user.

CSS specifies how to visually render HTML, not HTML itself. The only reason we can see HTML documents is because there's a default CSS stylesheet. Illustrations in the HTML spec are illustrative only. As far as a hypermedia system is concerned, HTML is for data.

Let's maybe wait for @jdesrosiers opinion to confirm if this explanation makes sense. This is his issue, after all. Or hit me up on the IRC channel.

[handrews]

@awwright

I don't believe "Resource selection" is a concept that makes any sense

Can you elaborate on this beyond just not liking my wording? I defined it at the top because I needed a name for the concept of resolving a URI from various sources (URI Template, instance data, user data). Please replace "resource selection" with whatever term you prefer and see if you can talk about the actual content of the proposal.

Let's try with just one specific question: You haven't addressed the idea that JSON Hyper-Schema is missing functionality at all- do you agree with that or not? You don't need to agree with "resource selection" as a term to answer that.

There's nothing in this proposal that talks about links restricting behavior. Please point to exactly what gives you the impression that there is.

I have no idea what point you're getting at with your discussion of web browser behavior. I used HTML as an analogy because you keep coming back to it. I needed something to provide an example of things that you can do with links with another hypermedia format that you cannot currently do with JSON Hyper-Schema. I also needed to reference it because it is the source of the "get"/"post" form terminology that you prefer. But there is nothing here that says JSON Hyper-Schema needs web browser behavior. Or that JSON Hyper-Schema needs to do specific things with links. Where, exactly, do you see that stated?

CSS doesn't have anything at all to do with it. I have no idea why you even brought it up, so if you want me to respond to that you're going to have to go into more detail.

[awwright]

@handrews We're sort of clobbering the author's issue, hit me up on IRC if you can (see the website)

[handrews]

@awwright I'll wait for @jdesrosiers to weigh in. Might make it all a moot point.

[handrews]

@jdesrosiers I can easily snip out all of the above and move it to a separate discussion if you prefer (@awwright I can follow up on IRC if that's the way we want to go for a separate discussion).

However, I put this all in this issue because I believe it advances us towards a better alternative to the confusing "get"/"post" language so I am interested in understanding if you (@jdesrosiers) find anything worth pursuing in it for that purpose.

[awwright]

@jdesrosiers Does my earlier explanation make sense?

[handrews]

@jdesrosiers any further thoughts here? I have put more of what I was trying to explore (none of which is addressed by @awwright 's comment) in #108 so we can remove stuff from this issue if you would like.

[jdesrosiers]

Sorry for disappearing for a while. I had to put this on pause to get through my other responsibilities. I have a few minutes for this at the moment, but my ability to contribute will be limited to non-existent for about another month.

@awwright, I understand the change you made and why you made it. In most ways, I think it's a good change. But, there is definitely a quality that is lost that I think is important. For example, as I type into this form, I don't know or care if the data is passed using GET or POST or whatever when I click "Comment". That detail is abstracted. I don't have to understand REST or HTTP to use github.com.

With the change to Hyper-Schema, a generic browser can't be created that abstracts knowledge of HTTP away from the user. The user needs to know and understand the semantics of the HTTP methods in order to use the API. Beyond that, they have to guess which methods are available or make extra calls like an OPTIONS request to find out. I would never have an issue like that with a web browser.

It should be possible for a Hyper-Schema driven API to have a completely generic browser that lets a user navigate it without knowldege of REST or HTTP just like user interaction with a web browser. Until that ability is restored, I consider Hyper-Schema broken.

However, I think adding the allowed https://github.com/json-schema-org/json-schema-spec/issues/73 keyword solves this problem. A generic Hyper-Schema browser can use that information to generate an interface that hides the details of HTTP and REST from the user.

@handrews, I look forward to looking into the other linking issues you commented on. I think it was the right idea to make it a separate issue.

[awwright]

@jdesrosiers

Why does a user need to understand HTTP to use JSON Schema? It works the same way HTML does. The whole point of the change is to enable generic user agents that can interact with JSON APIs, the same way user agents can interact with HTML today.

To some extent, yeah, the user agent needs to speak and understand HTTP. Recall that HTTP is an abstraction. So if I have a document with a link -- in HTML, or in JSON -- in my user agent, that shows up as underlined text, I click that text, and my user agent sends a GET request to the server for that resource. Likewise, if I have a form ("method": "post" in both HTML and JSON Schema), my user agent looks at the template (input elements with names, or a JSON Schema), displays form fields to let the user select values, and makes a POST request when I hit Return.

HTTP does abstraction so well, you never have to use an RPC to modify data - just standardized methods that mean the same thing everywhere.

So to help me understand where you're coming from:

1. Are there any differences between HTML and JSON Schema you see that I didn't cover here?
2. Do you have a specific example of something you're trying to accomplish?
3. Is there maybe a legacy system you need to interact with that violates HTTP's methods, e.g. you need to say "Hey, updating a value is actually done with GET here"?

[handrews]

shows up as underlined text, I click that text, and my user agent sends a GET request to

This, to me, is the abstraction of interest. The end-user does not see "send an HTTP GET" or even a URI. They see underlined text and know what that means. The user-agent does know about HTTP. In this perspective, HTTP is the concrete detail, not the abstraction. The presentation to the user is the abstraction, and the user-agent (in this case the browser) maps the abstraction to HTTP.

It's the same for forms that use POST. The browser abstracts away how HTTP is used to implement it, and instead just presents a specific interface to the user which has nothing of HTTP showing.

In API terms, this would be a generic client exposing some simple methods/functions that understand representations and URIs and do typical things with resources and links. That generic client would map from the simple methods to the HTTP requests/responses.

@jdesrosiers I think in this view, yes, the "user" needs to understand HTTP, but the "user" in this case should be the user-agent a.k.a. the generic client library. The "end-user" (application developer) does not need to know whether it's HTTP or something else underneath.

[awwright]

The user-agent does know about HTTP. In this perspective, HTTP is the concrete detail, not the abstraction.

The point of an abstraction is to expose a specific function and hide away the underlying detail. I don't think there's really such a thing as a concrete detail here. Or rather, the "concrete" detail is a bunch of chemical and electromagnetic states on a platter and electrons and photons traveling through copper, glass, and space. On top of that we build an abstraction -- CAT5/6/7/Wifi, Ethernet, IPv4/6, TCP, TLS. Every layer in that stack gives you some specific feature and hides another (TCP hides packet delivery and gives you reliable streams; TLS takes an arbitrary stream and gives you one that's secure; etc.)

Under this understanding of abstraction, HTTP is by definition an abstraction, it hides a reliable stream (usually either TCP or TLS), and exposes an interface for manipulating resources.

The presentation to the user is the abstraction

Right, in a sense. The user-agent is hiding an interface for manipulating resources (usually either HTTP, FTP, local filesystem, or CoAP) and presenting a UI for interacting with it.

---

I'm trying to figure out how this relates to the issue at hand. JSON Schema is entirely neutral on the issue of what protocol you're using. (There's a section on HTTP, yes; it describes conserns that should be self-evident, and exactly similar considerations apply for any other protocol.) There's plenty more hypermedia protocols than HTTP out there. These protocols define what you are to do in order to request a document, upload a document, or submit a document for evaluation; not JSON Schema.

Now, there's the question of: What if you want to indicate to a client which features another resource supports? e.g. What if you want to indicate that a particular resource is read-only?

Well first, this shouldn't ever be required. The canonical behavior is to just make an opportunistic request: If you want to persist a document, make a PUT request and if you can't, the server will tell you why (if the document is too large, it'll tell you; if the document is read-only, it'll tell you only GET is permitted); If you want to request a document, you make a GET request, and if you can't, the server will tell you why (if you need to authenticate, it'll tell you how; if the document moved, it'll tell you where).

But if you want to send this information in advance ahead of time, perhaps for performance reasons, there's certain metadata we can send informing the client of the nature of the resource. As far as I'm aware, though, there's not too many HTTP-specific link attributes. There are things like title, language, hash of the document, or media type; but those are properties of its contents, less so how to interact with it. I don't see anything wrong with defining such a link attribute (CoAP has plenty), but this just isn't the job of JSON Schema in particular: Anything I want to express via JSON Schema, I should be able to express with HTML or Atom or a Link header. They all have the exact same concept of links.

[handrews]

@awwright I was trying to reconcile your view and @jdesrosiers 's. Something can be an abstraction in one context and a concrete implementation in another.

From one perspective, HTTP is an abstraction that hides TLS/TCP and other lower-level details. This is your perspective.

From another perspective, a system such as HTML (and hopefully JSON Hyper-Schema) presents an interface to the user in a way that makes the interface's capabilities clear. This interface is an abstraction, and HTTP is the implementation detail. This, I believe, is the perspective that @jdesrosiers is coming from, and is a perspective that I find interesting as well.

I am not trying to prove you "wrong" about HTTP being an abstraction. It is. It's also a detail. It all depends on where you're standing as a user. To a user-agent programmer, HTTP is a vitally important abstraction. To an end-user, whether a browser user or an application developer, HTTP is a detail that they should not think about. Non-technical people use browsers all the time and have no idea what HTTP is or how it is used.

These concepts are complementary, they are not in conflict. But both perspectives are needed in order to understand the use cases that many of us have.

[Relequestual]

@jdesrosiers I understand this issue has stalled, and there has been lots of discussion around it, confusion, and general dissagrement in terms of what everything means and which expectations are OK and which are not.

Could I charge you to, after draft-6 is released (should be pretty soon I believe... @awwright ?), to repost this issue, summirising the various issues and comments raised, link to it from here, and then close in favour, please?

I mean, if you're not able to do this, I'll endevour to, but I can't say how quickly.

[jdesrosiers]

Sure, @Relequestual, I should be able to do that.

[handrews]

@jdesrosiers @Relequestual I think that since #292 removed "method" entirely and #294 is tracking the discussion around form semantics which was the unresolved aspect of that discussion, this issue can and should be closed. Any objections?

[jdesrosiers]

I agree that this can be closed. I don't agree that form is the last unresolved aspect of this discussion. Nothing has been resolved. Form is just the beginning, but the discussion in #294 is getting to the big picture I had in mind.