search

json-schema-org / json-schema-spec

The JSON Schema specification
http://json-schema.org/
Other
3.84k stars 267 forks source link

Finish and publish Draft 06 #170

Closed handrews closed 7 years ago

handrews commented 8 years ago

Hi folks, This brings my v6 scope notes out of the depths of the issue with all the comments and into its own tracking issue. I have expanded it with various issues and states at @Relequestual's request. @Julian this is what I referenced in the issue on the test repo the other day.

I'm going to treat this issue a little differently: We will no doubt have discussions in the comments, but I will keep updating this top comment with the current status as we work through things.

TL;DR Highlights

PRs for Draft 06 in priority order

meta-schema updates: #194 or #188, #189 or #185, #190

179 This hyper-schema issue is the big unknown of Draft 06. It addresses problems with "method", "schema" and "href" that really need to be at least partially addressed for Draft 06.

171 Wording for when keywords are absent, currently controversial. Touches a lot of language so let's get it resolved ASAP. (this is getting very close to resolution - yay!)

154 "id" to "$id" is just waiting for @awwright's approval or feedback. I am interpreting @epoberezkin's "thumbs up" on my "let's get this in Draft 06 and keep looking at other issues after that" as approval.

POSSIBLY IN scope for v6

173 The behavior of "deprecated", and for that matter the conceptual model for resources vs general instances in hyper-schema, is not clear enough to put this in Draft 06. However, if this can be resolved in time, it would be nice to include it.

OUT of scope for v6

129 Further work past #159, but may take a different form. Ignore for now.

163 Idea presented for comment, but almost certainly rejected.

Full Details

For most of the open PRs I noted what feedback, if any, the four currently most active people had provided (@awwright, @Relequestual, @epoberezkin, and myself), plus anyone else who has open concerns.

I added a line for tests where appropriate but have not looked into that at all yet. Obviously the new features need tests, and probably some of the clarified things could use regression tests as well.

NEW FEATURES

CLARIFICATIONS

BUG FIXES

epoberezkin commented 8 years ago

@handrews great summary, thank you. I will have a look over weekend.

awwright commented 8 years ago

I've been thinking it would be prudent to get as many new keywords in as possible, and any other meta-schema changes in, before publishing a new I-D.

I'm trying to find any other issues to tag under https://github.com/json-schema-org/json-schema-spec/milestone/2

I've pushed other issues out to a new "future" milestone. This includes one or two Hyper-schema meta-schema changes that should wait on other stuff first.

This list seems a little sparse though, if there's anything that's missing let me know.

epoberezkin commented 8 years ago

@awwright @handrews my only concern in that list is about #76. I thought the consensus was to do nothing about it and leave it as is. Because otherwise draft 6 will not be backward compatible and my understanding was that we want to have somewhat extended, clean and backward compatible I-D (as the previous one was a bit rushed-out). Am I wrong? Or is this list about draft-7?

handrews commented 8 years ago

@epoberezkin what about #76? It is not even clear what should be done from that one as there's not a clear push for "additionalPatternProperties" and there's no support for other things raised in that issue.

handrews commented 8 years ago

@awwright you're missing the most obvious one #159 and also #109 and #143 .

The point of the earlier discussion in #137 was to get Draft 06 out quickly due to the issues that the rest of us have with Draft 05. I don't want to get Draft 06 bogged down in a frantic push to shove as many features as possible into it. Isn't the point to do frequent drafts for feedback? It's better to get each successive draft out the door than to hold them as long as possible.

handrews commented 8 years ago

@awwright having considered this more and taken a look at your recent PRs I have some concerns:

  1. If you wanted to expand the scope of Draft 06, you could have easily spoken up in #137 last week after I posted the earlier version of this and @Relequestual and @epoberezkin both gave feedback. But instead of responding to our efforts to, among other things, accommodate your concerns about the pace of draft releases, you just expanded the work with no discussion.

  2. While "propertyNames" is probably fine to add to Draft 06 (I'd had it in the list until @epoberezkin asked for a fast rather than large release), "deprecated" is not ready and has not had enough discussion. Exactly how it fits into hyper-schema's notions of resources (already a bit unclear) vs fields is not obvious and we should not be holding up Draft 06 for it. Of course if we resolve it in the next day or so we can include it, but that seems unlikely right now.

  3. Please work on resolving the various open issues before expanding the scope of Draft 06. Most of them are blocked awaiting your approval or feedback, in some cases for multiple weeks with no acknowledgement from you. It has been very frustrating to watch you push back on getting reviews because you think it will slow us down too much, but then ignore PRs for days or weeks and add more work to Draft 06 when the rest of us are trying to wrap it up.

  4. @Relequestual discussed how he wanted to manage the issues and PRs here in #137 and has started making changes to the tags in the various repos. But you have now come up with an entirely different approach to managing things are are just doing that without discussion. And not quite aligning with what is here either, as noted in an earlier comment. We need to agree on a way to manage the draft process. Particularly when there is already a robust discussion among the rest of the community, you should not just run off and do something totally different. As with direct commits, when you take unilateral action on something that is under discussion within the community, you are showing a significant lack of respect for this community.

handrews commented 8 years ago

@awwright @Relequestual @epoberezkin I have updated the initial comment with a rough priority order for resolving the open PRs, focusing on getting the ones with the most uncertainty and/or broadest impact resolved quickly.

epoberezkin commented 8 years ago

what about #76? It is not even clear what should be done from that one as there's not a clear push for "additionalPatternProperties" and there's no support for other things raised in that issue.

@handrews exactly. I thought we would just drop it. I meant that I am concerned that it is added to the milestone (@awwright wrote: I'm trying to find any other issues to tag under https://github.com/json-schema-org/json-schema-spec/milestone/2).

handrews commented 8 years ago

@epoberezkin I have no idea what @awwright is doing with his milestones since he has not coordinated with what you, @Relequestual and I discussed in #137. As far as I am concerned, #76 is not in Draft 06 and we should not be hunting for more work. We should be finishing the open work that we already have in flight and getting a draft out.

Until at least the four of us can come to an agreement here, which I would like, on how we are managing this release, I am going to continue on the path that we discussed before. There are many ways that I would be perfectly happy to see the release managed, but we do need to agree on a single approach.

awwright commented 8 years ago

@epoberezkin I'm guessing #76 will be closed without making any changes, but it'd probably be a good idea to address the author's use case somehow. I'll continue discussion there.

Edit: Actually if that's the case I can just de-prioritize it by removing it from the "milestone"

awwright commented 8 years ago

@handrews What do you actually need from this issue?

If you'd like an I-D published, I can do that now, but I'd like to suggest holding off until most of the new/renamed keywords have been merged in. Maybe another week or two.

If you're waiting for a new feature to be merged in first, you can let me know in the respective issue.

handrews commented 8 years ago

@awwright this issue is defining the scope and tracking the progress for Draft 06.

Obviously I do not want a draft published right now. There is whole section of "PRs that need to be resolved" which, as you might guess, need to be resolved first. But I also do not want to keep tacking more and more stuff on. This is a very standard scoping exercise, is it really that hard to see?

I'm going to refer back to #137: You have repeatedly ignored the requests from the community for better project management. In response, you give your personal preferences, which we have all indicated we feel are inadequate to run the project. Yet you keep acting however you like instead of trying to work with us.

So I proposed something for scope and focus. I got feedback from @Relequestual and @epoberezkin and made some adjustments. @Relequestual suggested giving me write access to the repo so I could manage the issues and milestones, which apparently you are vetoing by just ignoring the suggestion entirely- I'm going to leave that decision up to the two of you and not discuss it further here.

Without write access, and therefore without the ability to manage milestones and labels, I'm working here in an issue to follow on the path that the three of us discussed. @Relequestual asked me to collect all of the PRs and issues involved in Draft 06, so I did that here.

But as you so often have, you just ignore what the rest of it do when you don't care for it, and tell us to do things your way. Which you don't explain very thoroughly, and change whenever it suits you.

If you want to facilitate JSON Schema in terms of project management, then you need to communicate clearly, consistently, and frequently. You need to actually listen to the rest of us and incorporate our feedback instead of just repeating the same explanations of why you're doing whatever you want. You should be able to tel by now that despite respecting your technical abilities, when it comes to the process here we're not buying your approach.

Either work to get us on board with how you plan to project manage (which is still not at all clear to me), or get involved in our discussions and work within the framework that we're attempting to produce through collaboration. Or at minimum, stop working at active cross-purposes.

I'm only attempting to do this because you have not. I don't need to be the project manager, but I have more time to devote to this project than (as far as I can tell) anyone else, and I need it to be managed by someone. I'm not claiming I have the best approach, but I have a clearly communicated approach that has gotten support from other active community members. If you have concerns about that, we can change that, but you need to convince us and get clear support from us rather than just doing what you've been doing.

awwright commented 8 years ago

@handrews I'm sorry if progress forward isn't going as fast as you like. I'm actively trying to moderate that and other expectations, even when it's not the most popular thing to do.

The IETF is very clear on what an Internet-Draft may be used for, and those are the constraints I have to follow as document editor, it's the same process used by all the other groups doing the same thing. Anguish with this process is not unfamiliar to me, I understand! I've literally been laughed at for suggesting some of the things you have.

Overall, I'm trying to accommodate as many requests as possible, but if there's a lot coming in, that necessarily means that issues have to languish. My apologies if that looks like I'm ignoring them!

Right now my plans for I-D are to wait until most of the features in https://github.com/json-schema-org/json-schema-spec/milestone/2 are checked off. If you would like to see something different, by all means let me know, but this is a rather verbose issue and I'm not exactly sure what changes it's proposing.

I also want to note you're posting very lengthy issues in general that I'm doing my best to work with. I want to keep issues on-topic, please drop me an email or meet up in IRC.

handrews commented 8 years ago

I'm sorry if progress forward isn't going as fast as you like.

@awwright it has nothing to do with speed and everything to do with you not listening, as you just demonstrated with this response.

My problem is that you make this all about you. It's all about what you will do when you will do it, and what you have decided is what the project will do. It's never about "us" (the community, not you and me specifically). @Relequestual and @epoberezkin have been much more collaborative, and that is even with @epoberezkin and I vehemently disagreeing on technical points much of the time. I feel like the three of us are all trying to move together in roughly the same direction (although I invite either of the other two to object if I am mischaracterizing their views).

You, on the other hand, are doing your own thing and frequently surprising everyone with decisions out of the blue, or duplicated work, or work that crosses other work going on.

If you want to be "the leader" (in quotes because theoretically this is a community project and not owned by a single leader), it is your responsibility to stay on top of the project. This issue is specifically for managing the large incoming workload and backlog, so when you say you're trying to balance that but at the same time ignoring all efforts by anyone else to bring clarity and focus to that exact problem, I have no sympathy for your position. If you do not have time to keep on top of everything and manage expectations, then let someone else do it. The document editor/technical lead and the project manager don't need to be the same person. In most organizations they are not.

awwright commented 8 years ago

@handrews I'm afraid there's indeed been a lot of times I have to inform everyone it's not possible for us to do things a certain way. To that end, I try very hard to separate my personal views from my role in editing the document.

Please see me on IRC if you want to discuss specifics, issues and other long-form responses tend to be exceptionally poor at pegging down specific misunderstandings.

handrews commented 8 years ago

@awwright While we worked out some things on IRC, I want to follow up here just a bit.

I'm afraid there's indeed been a lot of times I have to inform everyone it's not possible for us to do things a certain way.

I still don't know why you said this. All I have asked for is that we agree on a process within the community and all follow that process. While I have suggested some options for a process, I have not demanded any specific solution. So what, exactly, is not possible?

handrews commented 8 years ago

@awwright to follow up on the discussion of how to close this that we were having in IRC when my network connection dropped, here are the discrepancies that I see with the current milestone assignments:

55 should move from "future" to "draft-6", as there's a simple PR open that will let me close it out.

178 and #169 are tiny one-line clarifications and should be in "draft-6"

168 (meta-schema updates) should be in draft-6, as it's stuff the meta-schema needs sorted (I have partially implemented the fixes that we discussed yesterday, and will update the PR probably tomorrow)

PRs #154 and #167 should be in draft-6 (their issues, #20 and #101 respectively, already are, but we should be consistent about labeling the PRs as well)

150 ("uritemplate" and "jsonpointer" formats, corresponding issue is #109) should either be in draft-6, or needs a comment from you explaining why it should be held off.

167 (boolean schemas) should be in draft-6. I notice you have the issue (#101) but not the PR (#167).

I have my concerns over whether #173 is ready for draft-6, but I mentioned them there and it can certainly stay in draft-6 unless/until we come to a decision on it.

handrews commented 8 years ago

Beyond that, @Relequestual may have other things he wants from this, but once we agree on a set of issues/PRs for draft-6 I don't need this to stay open.

We do need to figure out where to discuss any major shifts in draft-6 contents, and discuss what should be the focus in draft-7 (we don't need to do that right now). I suppose for draft-6 if we don't want to keep using this issue, we can file a new one if we encounter anything that makes us reconsider the overall scope. Otherwise requests to change milestones can be put in comments of the relevant issues.

epoberezkin commented 8 years ago

@handrews here it goes:

Features

Boolean subschemas:

128 - ok in general, with minor corrections

167 - not ok, I am in favour of #128, I think allowing root schema to be a boolean is a bad idea.

$id

154 - not convinced we need to drag around deprecated id. I would just drop it...

159 - didn't review. I don't go in hyper-schemas...

propertyNames

172 - ok. With whatever language for "defaults to empty schema" (or we can say true, given #128)

jsonpointer and uritemplate

150 - ok, see comment.

exclusiveMaximum

168 - ok

162 - I disagree. I see no harm in keeping boolean form. It has use cases as I tried to explain.

examples

123 - I think it should be "MUST be valid". see comment

169 - ok

contains

113 - ok

const

139 - ok

Empty array

112 - ok.

168 - why items array can't be empty?

Clarifications

May be considered

171 - I disagree, it became less clear. See comments in PR

items/additionalItems

156 - ok

items consistent with properties

111 - I am not a fan of this "child instance validation" language to be honest. I think "items validates array" language was better. But ok.

principles

143 - ok

meta-schema URIs

164 - isn't it old?

CBOR

153 - as usual, all the objections in #6 were ignored by @awwright, #153 was submitted and merged straight away on the following day without any discussion. I can repeat, I think that this spec should not be in any way concerned with representation formats even it helps lobbying the usage of JSON Schema. It will backfire at some point as all alternative formats are extensions of JSON and by endorsing their usage in the spec you create false expectations that they would work without being clear on what are the limitations.

Fixes

integer

147 - ok

additionalItems

111 - see comment above

typos

166 - ok

epoberezkin commented 8 years ago

with @epoberezkin and I vehemently disagreeing on technical points much of the time

@handrews I'd say 30% of the time, to be precise ;)

handrews commented 8 years ago

168 - why items array can't be empty?

It can- there is no "minItems" in the "schemaArray" definition.

epoberezkin commented 8 years ago

@handrews see https://github.com/json-schema-org/json-schema-spec/pull/168/files#r90764468

epoberezkin commented 7 years ago

I keep thinking that for consistency we should use hyphens in new formats names, as we have in "date-time". "json-pointer", "uri-ref", etc.

@handrews any chance you reconsider?

handrews commented 7 years ago

I hate the hyphen in date-time, so since it was already inconsistent I went with uriref because I like it better and one of the things I was introducing was a uri thing anyway.

If you can get support for changing all of them (date-time, uri-ref, uri-template, json-pointer, I dunno, host-name? blech) then I won't block it. But I'm not otherwise going to spend time thinking about this.

epoberezkin commented 7 years ago

I have a got feeling that we can end up supporting both versions, with and without hyphen :)

But whatever, indeed.

handrews commented 7 years ago

Now that @Relequestual has made the new label system and I am able to use them, I think there is no longer value in keeping this epic discussion open. I will make a checklist issue for things that need to happen for Draft 6 publication, but we don't need all of this detail.

Anyone who was trying to get anything done here other than track Draft 6, please file your own issues :-)

handrews commented 7 years ago

Opened #206 to cover the parts of this that are actually important right now.