search

dret / webconcepts

Web Concepts: Concepts that matter for the the Web.
The Unlicense
141 stars 29 forks source link

JSON Format for each concept value #25

Closed arno-di-loreto closed 7 years ago

arno-di-loreto commented 7 years ago

We can get all concepts in JSON with this URL http://webconcepts.info/concepts/concepts.json, but it would be a great addition to be able to access each concept value individually as a json file. For example: http://webconcepts.info/concepts/http-status-code/409.json

dret commented 7 years ago

please review and close if you're happy.

arno-di-loreto commented 7 years ago

Perfect! Thanks Erik!

dret commented 7 years ago

keep the requests coming! i am particularly interested in ideas what to cover beyond the concepts currently listed here: http://webconcepts.info/concepts/

dret commented 7 years ago

@mitraman voiced concerns over using the concept name as the object key in the concepts JSON, saying that would be harder to parse. i am wondering what my current JSON beta users think about that? @kinlane @arno-di-loreto

kinlane commented 7 years ago

I agree with him....that it would add some overhead. I flip flop on this type of thing....one week I'd feel it adds overhead, while the other I think it defines it better and allows you to be more precise in code....in the end I'm 50/50 about it. ;-0

dret commented 7 years ago

so be be more concrete, the current version is this:

{
   "404": {
    "id": "http://webconcepts.info/concepts/http-status-code/404",
    "description": "Not Found",
    "details": [{
     "description": "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. A 404 status code does not indicate whether this lack of representation is temporary or permanent; the 410 (Gone) status code is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent.",
     "documentation": "http://tools.ietf.org/html/rfc7231#section-6.5.4",
     "specification": "http://webconcepts.info/specs/IETF/RFC/7231" }]}}

i think (but i may be wrong) that @mitraman would prefer something like this:

{
   "concept": {
    "name" : "404",
    "id": "http://webconcepts.info/concepts/http-status-code/404",
    "description": "Not Found",
    "details": [{
     "description": "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. A 404 status code does not indicate whether this lack of representation is temporary or permanent; the 410 (Gone) status code is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent.",
     "documentation": "http://tools.ietf.org/html/rfc7231#section-6.5.4",
     "specification": "http://webconcepts.info/specs/IETF/RFC/7231" }]}}

and while we're at it, we could for good measure also add the concept "type", so that it makes more sense as a standalone JSON:

{
   "concept": {
    "name" : "404",
    "type" : "http-status-code",
    "id": "http://webconcepts.info/concepts/http-status-code/404",
    "description": "Not Found",
    "details": [{
     "description": "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. A 404 status code does not indicate whether this lack of representation is temporary or permanent; the 410 (Gone) status code is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent.",
     "documentation": "http://tools.ietf.org/html/rfc7231#section-6.5.4",
     "specification": "http://webconcepts.info/specs/IETF/RFC/7231" }]}}

if we go this route, then i think for consistency the specs.json and concepts.json files should follow the same conventions. that would make them a bit bigger, but if that seems to be more in line with what people expect, then i'd be more than happy to make this change.

arno-di-loreto commented 7 years ago

Going that way, having in mind a (hypermedia) API providing webconcepts information and given that these files are what you want to provide to end users, I'd simply remove the "concept" key into individual files:

{
    "name" : "404",
    "type" : "http-status-code",
    "id": "http://webconcepts.info/concepts/http-status-code/404",
    "description": "Not Found",
    "details": [{
     "description": "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. A 404 status code does not indicate whether this lack of representation is temporary or permanent; the 410 (Gone) status code is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent.",
     "documentation": "http://tools.ietf.org/html/rfc7231#section-6.5.4",
     "specification": "http://webconcepts.info/specs/IETF/RFC/7231" }]
}

And in the list file (concepts.json), I would replace the map with an array:

Before:

{
 "http-cache-directive": { ... },
 "http-status-code": { ... }
}

After:

[
 { /* http-cache-directive data */ },
 { /* http-status-code */}
]

An alternative could be having the array in a property (so later you can add all the hypermedia things):

{
  "items": [
   { /* http-cache-directive data */ },
   { /* http-status-code */}
  ]
}

The API Stylebook API is designed this way, it's not yet available (waiting for DNS refresh) but you can see the json files in this repo apistylebook-data

arno-di-loreto commented 7 years ago

DNS gods be praised! http://api.apistylebook.com/design/topics.json You, Kin and Ronnie have private access to this alpha version of the API Stylebook API 😉

dret commented 7 years ago

thanks for suggestions and access, i'll have a look later today or tomorrow. @ronnie, would those changes make you happy? (and as usual, it really sucks that JSON has no comments!)

dret commented 7 years ago

btw, just another question since i'll have to refactor the JSON generation anyway: would it make sense to also separate generate JSON for each individual concept? so that for example there would be one JSON just containing http://webconcepts.info/concepts/http-headers, so to speak? that right now is a gap between the "all concepts" JSON, and the individual JSON for each concept value that started this issue.

mitraman commented 7 years ago

I'm not overly worried about the JSON format as long as the data is easy for me to find.

The reason I suggest an explicit name property is that I expect to grab these concepts based on a URL reference. My tool isn't going to know it's looking for a 404 - it just knows it needs to pickup a concept from webconcepts.info/rfcnumber/404. So having a property in the JSON named "404" isn't helpful.

In other words - my main use case is a generic one, so I want data that uses consistent, generically useful property names.

The basic format that @arno-di-loreto suggests works for me. Maybe some additional discussion is needed on how to represent the relationships.

dret commented 7 years ago

On 2016-09-21 16:35, Ronnie Mitra wrote:

The basic format that @arno-di-loreto https://github.com/arno-di-loreto suggests works for me. Maybe some additional discussion is needed on how to represent the relationships.

absolutely; this is a very good point. because in the end that's about the question how concepts and values are identified, so that they then can be related via those identifiers. because this is a pretty fundamental issue (and probably one that should be resolved better sooner than later), this is tracked separately in #24.

dret commented 7 years ago

new JSON ready for review... http://webconcepts.info/update/2016/09/22/JSON-refactoring.html curious to hear what you guys think. and sorry @kinlane for breaking your stuff. but i thought @mitraman had a point and wanted to make the JSON more self-contained so that smaller snippets still make sense. which makes #24 an important thing to get right...

kinlane commented 7 years ago

No breakage on my end. I'm still just playing with possibilities. I will provide any pros / cons regarding effect of my tooling.

dret commented 7 years ago

On Sep 23, 2016, at 19:53, Kin Lane notifications@github.com wrote: No breakage on my end. I'm still just playing with possibilities. I will provide any pros / cons regarding effect of my tooling.

great, thanks! so if it works good enough for all, i guess we can go ahead and close this one. thanks all for the feedback!

arno-di-loreto commented 7 years ago

First thoughts after only looking at the 2 examples provided in the blog post (I'll give a deeper look by trying to use the web concepts data with apistylebook.com):

After thought: maybe I should create a new issue?

dret commented 7 years ago

On 2016-09-24 10:05, Arnaud Lauret wrote:

First thoughts after only looking at the 2 examples provided in the blog post (I'll give a deeper look by trying to use the web concepts data with apistylebook.com):

  • I like having id as URL because it makes the data (almost) browsable even without having choosen an hypermedia type (almost because I have to add ".json" to new request in Postman :-)

the wonders of web architecture: use URIs as IDs and make them resolvable.

  • I LOVE the name-singular and name-plural properties, it's a brilliant idea!

thanks. driven by my requirement to generate the web site from my data.

yes, this is correct. there is a technical reason for this which is that jekyll sucks. but in the end it shouldn't matter this much if everything is interlinked, right? i agree that it would be nice to have the same path segment, but it would only work if the concept URI would have a trailing slash http://webconcepts.info/concepts/http-headers/, and i hate those. or are you a jekyll ninja and there's some way around this? i couldn't find one, but i am not a jekyll expert.

i see your point. not sure i like it as the URIs get longer and i like shorter ones. but yes, you now could add additional sub-resources to concepts. i am not quite sure which these ones would be. the /rfcs would just be a list of RFCs contributing to this particular concept? i could easily generate this, but i am not quite sure how useful that would be.

so yes please, @mitraman and @kinlane, let us know what you think. i am slightly in favor of not doing this, but i think i could be persuaded rather easily if most think that's a potentially useful thing to do.

  • What is the purpose of the |details| array on a concept value? , it seems to be both used to describe the value and list references to this value in RFC's, what item I'm supposed to choose in the array as the value's "official" description?

ah, you spotted a weakness. good job. there are not many values with more than one entry there. and truth be told, webconcepts currently has no info which one is the "official" one. the reason for this is that IANA has no actual info on this as well: you have to look through the specs that claim to define the concept and then decide which one you decide is most authoritative. to me, that's a conceptual weakness of how IANA manages this.

but i see that this is not very helpful. all i can offer is:

what do you think?

  • Maybe having a description (or even details array) on concept would be interesting too

good point. currently all i have (i think) is the link to the IANA registry (is there is one). what other metadata are you thinking about?

  • And I as was reading http://webconcepts.info/concepts/http-headers.json, I discovered the ALPN header which must be included only in a HTTP CONNECT request: would it be relevant and possible to add references to other concepts or concept's values? In this case, the http-header ALPN having a link to the http-method concept's value CONNECT.

that's a rathole i'd rather not go down. there are many variations of this. a more semi-official one is whether HTTP header fields and request-only, response-only, or both, and there's interesting side cases here as well.

my usual mantra is to keep models as shallow as you can get away with. i hope i can get away with not modelling all the delicate relationships that concept values have, because i am pretty sure that these will be very hard to generalize, and very hard to correctly extract from the language of the specs, which will have no notion of our glorious ubermodel of how web concepts can interrelate.

dret commented 7 years ago

i am working on factoring in both @arno-di-loreto's and @mitraman's feedback. this may break stuff on your end, but in the end should result in a better hierarchy of URIs (as suggested by @arno-di-loreto) and more straightforward URIs (@mitraman). maybe this issue is a bit much for tracking all these things. if you do have issues after the refactoring, please open separate issues. thanks!

dret commented 7 years ago

some of the things i have broken so far (more to come):

dret commented 7 years ago

after trying to address @arno-di-loreto's issue for a while, i had to give up. http://stackoverflow.com/questions/41113834/can-jekyll-serve-files-when-a-directory-of-the-same-name-exists is my problem, and if anybody knows how to solve this for jekyll, i'd be very happy to hear about it. thanks!

dret commented 7 years ago

all hail stackoverflow! http://stackoverflow.com/questions/41113834/can-jekyll-serve-files-when-a-directory-of-the-same-name-exists/41137216 now has an answer and i am going to implement and test that. if this works, this means URIs are going to change (in the JSON data as well as on the web site), potentially breaking code that's out there. after this is done, i am going to close this issue because that will address the main issue. @arno-di-loreto, if you still want some of your other concerns to be addressed, please raise individual issues (#28 already addresses one of your concerns: the fact that multiple definitions/details may exist). thanks!

dret commented 7 years ago

http://webconcepts.info/update/2016/12/15/hackable-concept-URIs.html summarizes the changes that went live with the last commits. this closes this issue.

dret commented 7 years ago

http://webconcepts.info/JSON is some documentation for the concept JSON format.