RA API specification in -next with small extensions

[kgaillot]

What do you think about using a directory structure like:

ra/
   1.0/
   current -> 1.0
   next/

When we are ready to propose a 1.1, we could copy the proposed items from next into a new 1.1 directory, and when 1.1 is approved, re-link current to it.

That would give people reliable URLs to link to, either for a particular version of the standard, or the latest. It would also allow us to have version-specific documentation, examples, etc., in each subdirectory.

[marxsk]

I don't know if we need more than two versions (excluding current situation). It is a valid idea that various versions should be linkable but this can be done via tags on github quiet easily. IMHO I don't expect too many releases (1.1; 2.0) and then we don't have more ideas.

In general, I do not care too much about -next and -current, so it can be easily replaced by directories. My main goal is to get at least 1.1 ready before pacemaker 2.0 :)

[kgaillot]

I think it could be useful to have all formally adopted versions in the main branch. One possibility that's been discussed is promoting OCF to its own installed package; ocf-doc would contain the text of the standards, and it could have all of them. This would allow people to look at the version tag in an OCF agent and then compare it against the relevant standard, regardless of whether it's the most current (or similarly, validate a particular agent against the appropriate RNG schema).

[marxsk]

ok, so my proposal will be a separate file. What should be a filename?

[kgaillot]

ok, so my proposal will be a separate file. What should be a filename?

I'm guessing you're thinking something like this (example after 1.1 is released):

ra/
   ra-api-1.0.rng
   ra-api-1.1.rng
   ra-api-next.rng
   ra-api-latest.rng -> ra-api-1.1.rng
   resource-agent-api-1.0.md
   resource-agent-api-1.1.md
   resource-agent-api-next.md
   resource-agent-api-latest.md -> resource-agent-api-1.1.md

That would be fine, or we could use directories for each version, e.g.

ra/
   1.0/
      ra-api.rng
      resource-agent-api.md
   1.1/
      ra-api.rng
      resource-agent-api.md
   next/
      ra-api.rng
      resource-agent-api.md
   latest -> 1.1

Regarding the file names themselves, I think it makes sense to stick with "resource-agent-api" for both, and just vary the extension.

[marxsk]

I agree with directories and 'next'. So the old one will be marked as 1.0 even that it was never released. Right?

[kgaillot]

I agree with directories and 'next'. So the old one will be marked as 1.0 even that it was never released. Right?

Yes, we have to make some concessions to history. :-)

[jnpkrn]

My take on this matter would be:

• make a distinction between machine readable formalism + its example instances if applicable vs. the rest (guide/tutorial/commentary, auxiliary files with constant definitions, the former will be tracked with per-minor granularity, the latter with per-major granularity -- properly documenting the additions, i.e. the only allowed change amongst between subsequent minor versions, and possibly factoring the common grounds out of the versioning concept)

• based on the previous and building on Ken's idea, make the dir structure like this:

  ra/
     api-1/
        1.0/
           ra-api.rng
           ra-api.example
        1.1/
           ra-api.rng
           ra-api.example  # should use new features from 1.1,
                           # otherwise be a mere symlink to 1.0
        resource-agent-api.md
     api-2/
        2.0/
           ra-api.rng
           ra-api.example
        next/              # start modest, if incompatible changes
                           # emerge, locate it as api-3/next
                           # instead, bumping "next" symlink
                           # at the parent along
           ra-api.rng
           ra-api.example
        resource-agent-api.md
     latest-api -> latest-api-2
     latest-api-2 -> api-2/2.0
     next -> api-2/next
     resource-agent-api-common.md

[jnpkrn]

On 06/12/17 17:57 +0000, Ken Gaillot wrote:

I think it could be useful to have all formally adopted versions in the main branch.

If ever/when the project is to get first-class distributable deliverables (see below), I think it would be preferred to cater both its slow- and fast-moving consumers, resulting in a need to ship more versions in parallel, and for that, branch based iterations would be impractical (tarball recipe requiring checkouts from multiple branches, incompatibility with on-the-fly archives from a repository snapshot).

One possibility that's been discussed is promoting OCF to its own installed package; ocf-doc would contain the text of the standards, and it could have all of them.

To this end, a proposal to promote this project to a first class entity in the component graph, making it implementation-less holder of codified constants and hence sort of a fixed point to be depended on in such graph (allowing such conceptual dependants to become actual dependants, eliminating redundancy), is upcoming.

-- Jan (Poki)

[marxsk]

@jnpkrn

ad machine readable/rest) I do not expect that there will be other documentation in the rest. But, I agree that it is enough to have documentation for major releases.

ad directory structure) I'm not sure if I understand the benefits of such granularity. I would remove api-[12] directories as well as latest-api*. AFAIK people will just use their version for as long as they want and more often than not event that won't be true and they will use their own tweaked versions because of existing deployments.

I expect that now we are ready for 1.0 and we are almost ready for 1.1. After this branching stuff, I will prepare 2.0 pull requests. And we can expect that we will miss something, so 2.1 might be released in 2018 as well. But as both resource/fence agents are so stable we did not have a need to introduce new features. If other people will start to use that part of the standard, the situation might change, of course.

ad) first-class deliverables It sounds interesting but I'm not sure about possible users.

@kgaillot Can you please make a final decision about directory structure based on poki comments? I'm happy with your proposal.

[jnpkrn]

On 08/12/17 11:16 +0000, Marek Grác wrote:

ad directory structure) I'm not sure if I understand the benefits of such granularity. I would remove api-[12] directories as well as latest-api*. AFAIK people will just use their version for as long as they want

Do you mean authors for the software pieces that incline to use some bits of OCF and that bundling will be common? No wonder that's the current state but I want to believe we want to change that, going forward.

and more often than not event that won't be true and they will use their own tweaked versions because of existing deployments.

Again, I want to believe that it's in the best interest of all the parties to unify theory and common practise under single point of authority.

I expect that now we are ready for 1.0 and we are almost ready for 1.1. After this branching stuff, I will prepare 2.0 pull requests. And we can expect that we will miss something, so 2.1 might be released in 2018 as well. But as both resource/fence agents are so stable we did not have a need to introduce new features.

I would define readiness by terms of having the redistributable, useful content (see the bottom of my previous comment) ready, than it makes sense to proceed with tagging the release.

If other people will start to use that part of the standard, the situation might change, of course.

ad) first-class deliverables It sounds interesting but I'm not sure about possible users.

Resource managers, for a start? Resource agents to refer there to fetch the basic, standard-related constants?

That would be a way forward from the current bundling/non-systemic, redundant and not necessarily parity maintaining mess. At least in my view.

-- Jan (Poki)

[marxsk]

2017-12-08 14:31 GMT+01:00 Jan Pokorný notifications@github.com:

On 08/12/17 11:16 +0000, Marek Grác wrote:

ad directory structure) I'm not sure if I understand the benefits of such granularity. I would remove api-[12] directories as well as latest-api*. AFAIK people will just use their version for as long as they want

Do you mean authors for the software pieces that incline to use some bits of OCF and that bundling will be common? No wonder that's the current state but I want to believe we want to change that, going forward.

Probably this is a conceptual problem. I try to solve current issues instead of those that might happens in long term future. I do not expect people to bundle OCF standard to end-user application as it have only limited benefit (if any). On the other side, it might be good to have discussion about where we want to head with OCF standard. I focus on near future as this can solve my issues with OCF and afterwards AFAIK there won't be any serious issues (according to discussions on summit).

and more often than not event that won't be true and they will use their own tweaked versions because of existing deployments.

Again, I want to believe that it's in the best interest of all the parties to unify theory and common practise under single point of authority.

Yes, it is the best interest but there is no chance that vendors will switch their products with really long support. There is no single point of authority (and there will never be). If any important group decides that they need extension, they will do it. And hopefully, they will attempt to standardize it afterwards.

If other people will start to use that part of the standard, the situation might change, of course.

ad) first-class deliverables It sounds interesting but I'm not sure about possible users.

Resource managers, for a start?

AFAIK there is one user outside of 'default' HA cluster world. And there is no need for systemd to use our standard, it is more likely that we will switch to theirs one day.

Resource agents to refer there to fetch the basic, standard-related constants?

I'm not aware of any such constant that might be useful. @oalbrigt might know more.

That would be a way forward from the current bundling/non-systemic, redundant and not necessarily parity maintaining mess. At least in my view.

Yes, it will but I prefer to find 'customers' before such large changes.

[[just sidenote]] I have several attempts with OCF before and as this approach was not flexibile enough, I have decided to create our own standard for fence-agents (which are not standardizied by OCF at all yet - I'll just create a patch on top of 2.0). The only reason, why I want to standardize some extensions is to simplify UI building as Ken don't like the idea to use anything other than OCF standard in pacemaker. And I want them in pacemaker :)

[kgaillot]

, the former will be tracked with per-minor granularity, the latter with per-major granularity

Everything, including documentation and examples, will have per-minor granularity, so I don't see a need for major-version directories. Minor versions will introduce significant new syntax that needs to be documented. Major versions are strictly for backward-incompatible changes, which hopefully will be extremely rare.

I could see maybe having major-version symlinks, like

ra/1.0/ ra/1.1/ ra/2.0/ ra/latest -> 2.0 ra/1 -> 1.1 ra/2 -> 2.0

in case someone wants to link to e.g. "the latest version 1 standard".

If we also want a "shared"/"common"/"include" directory for convenience of reusing text as part of the (future) build process, that could make sense, too, but it would not be part of the released standard itself, so I think we can leave it for later.

I expect that now we are ready for 1.0 and we are almost ready for 1.1. After this branching stuff, I will prepare 2.0 pull requests. And we can expect that we will miss something, so 2.1 might be released in 2018 as well. But as both resource/fence agents are so stable we did not have a need to introduce new features. If other people will start to use that part of the standard, the situation might change, of course.

If possible, we should avoid a 2.0. If we're just making compatible additions, or deprecating but retaining support for old syntax, it won't be necessary. I sort of would like to see a road map like:

1.0: remove the "DRAFT" notice from the top of the standard and officially release it after nearly 20 years :-)

1.1: codify all the de-facto changes that have arisen in the years since, and release that, simply to recognize it, and have an easily agreed on base to work from. this wouldn't be useful to automated interpreters since agents may report 1.0 or 1.1 interchangeably, but it will be an official record of the changes.

1.2+: all the changes we'd like to see

ad) first-class deliverables It sounds interesting but I'm not sure about possible users.

The idea is for some of the stuff currently in pacemaker, pcs, resource-agents, and fence-agents to be moved to a common ocf package that they can share. The most obviously useful item is the set of OCF exit status codes, but I could see going further and moving some library code (e.g. ocf-shellfuncs) here as a reference implementation. Perhaps one day the project could build ocf-libs (C/shell), ocf-doc (text of the standard itself), and ocf-devel (C headers).

[marxsk]

@kgaillot

thanks, I will create a new PR that will follow those directories

Although, I agree with you about 1.x branch. I believe that we need 2.0 to be able to break backward compatibility but we can postpone this discussion a bit :) The main reason is to remove parts that allows you to put everything inside (e.g. HTML/XML markup). I don't believe that it was used at all but it is change that breaks backwards compatibility.

[jnpkrn]

On 08/12/17 08:18 -0800, Ken Gaillot wrote:

, the former will be tracked with per-minor granularity, the latter with per-major granularity

Everything, including documentation and examples, will have per-minor granularity

I should have been told this whole discussion is moot since everything has been decided and any attempt to reshape the proposal is futile

Anyway, I think everything-in-git-structure and everything-in-subdirs are two extremes and my proposal tries to find a best balance between them:

• for machines, new formalism would need to be introduced to capture additions if all intra-major changes would be iteratively merged and the distinction between them would still be desired -> not easy (or cumbersome at best if plain differentials would be distributed)

• for humans, programmers, it's customary to work with "API additions" directly in the documentation, allowing to get rid of the redundancy and the hassle of finding out what was added when -> easy (just remember to add marks per convention to denote this new thing is only applicable since version X.Y)

so I don't see a need for major-version directories. Minor versions will introduce significant new syntax that needs to be documented.

No contradiction with the rolling updates to the documentation part within particular major release.

Major versions are strictly for backward-incompatible changes, which hopefully will be extremely rare.

Which would then justify starting the documentation cleaned from historical ballast for this major jumps.

-- Jan (Poki)

[kgaillot]

Everything, including documentation and examples, will have per-minor granularity I should have been told this whole discussion is moot since everything has been decided and any attempt to reshape the proposal is futile

That statement was an observation, not a decision. We'll need examples that show the features of a particular minor revision, and they won't be applicable to all revisions. Similarly for documentation; I hadn't considered the "since version X" approach, but I don't think it's appropriate for a standard. I think versions of a standard should be completely self-contained.

[jnpkrn]

On 08/12/17 12:19 -0800, Ken Gaillot wrote:

We'll need examples that show the features of a particular minor revision, and they won't be applicable to all revisions. Similarly for documentation; I hadn't considered the "since version X" approach, but I don't think it's appropriate for a standard. I think versions of a standard should be completely self-contained.

Perhaps not appropriate for widely spread normative standards such as HTML, but even that bothers to enumerate the changes explicitly in its appendix: https://www.w3.org/TR/html401/appendix/changes.html#h-A.1

And effectively, there's not much of a difference compared to indicating the addition directly in place, and then there's next to none point to maintain a completely separate document.

Afterall, we are discussing sort of an API, and its customary to add new extensions, properly anchored to the introducing version, to keep the document one-off stop regardless of particular compatible version being in question. This approach is what I'd like to see applied here as well -- one document per major version.

-- Jan (Poki)