search

kubernetes / website

Kubernetes website and documentation repo:
https://kubernetes.io
Creative Commons Attribution 4.0 International
4.44k stars 14.28k forks source link

Migrate to updated API reference generator #25505

Open sftim opened 3 years ago

sftim commented 3 years ago

This is a Feature Request

What would you like to be added k-sigs/reference-docs has a recently-added gen-resourcesdocs generator. Credit to @feloy for hard work on this.

Let's switch to using that generator [Preview] and make it the default.

This entails:

Why is this needed

The existing API documentation (example) uses a different style from the rest of the website, is harder to navigate than it could be, and is also difficult to localize.

Whilst it has served the Kubernetes project well, with the arrival of an alternative generation option it is time to look at switching.

Comments

/kind feature

/area web-development (sic)

also /language en

Relevant to issue #24441 and to https://github.com/kubernetes/kubernetes/issues/19680

kbhawkey commented 3 years ago

Is there a suggested timeline for making the proposed changes? There are quite a few (quick search found ~828) links that refer to sections in the single page reference. I suspect that the new shortcode could help create replacement links to the appropriate resources?

sftim commented 3 years ago

My sketch for part of an idea: replace the current API reference with a page that uses JavaScript to calculate a redirect to the correct URL (based on the fragment identifier and some other information). That JavaScript could either XMLHTTPRequest the real API reference pages, or use information available at site build time.

That would allow us to update existing inbound links gradually, including across the 12 downstream localizations. It would I think take that clean up away from the critical path for migrating, too.

For a long term migration away from that we could make a whole new redirection service eg https://api.k8s.io/example.k8s.io/v1beta1/KindOfObjectGoesHere - but we don't need to do that immediately, or even at all.

kbhawkey commented 3 years ago

I was thinking that the root path for the API reference pages could serve as the new API reference URL (add a redirect)? Are you suggesting to dynamically redirect every path off this base URL?

Perhaps the static assets and single HTML page can remain until every localization has updated their files. The en content could start to use the shortcode to reference the resource pages (if every API ref link can be resolved by using the new shortcode).

tengqm commented 3 years ago

Should we consider the coexistence of the two API reference representations for one or two release cycles? It is a common practice for websites to serve two flavors of some contents till a complete switch over proves to be welcomed.

sftim commented 3 years ago

one or two release cycles?

I really would prefer not to break these URIs because they are referenced in code and documentation all over the web. That's why I really like the idea of a long lived helper that redirects to the new home. We could redirect everything to one place but that's not much different from a 404 page; if someone has a link to the API for ConfigMap, then what they need to see is the API reference for ConfigMap. Otherwise they may as well ask a search engine. (which right now will probably serve https://docs.openshift.com/container-platform/3.7/rest_api/api/v1.ConfigMap.html as the top result, FWIW)

kbhawkey commented 3 years ago

Should we consider the coexistence of the two API reference representations for one or two release cycles? It is a common practice for websites to serve two flavors of some contents till a complete switch over proves to be welcomed.

Yes, that seems reasonable. I have not thought about the timeline for completely switching over. I suggest to decide on a timeline and set up a date to communicate this decision to the community (how and when these changes will finally happen). There may be issues logged. I see a short list of things to do:

  1. Update the intra-site links to refer to the new API reference pages.
  2. Determine how to redirect external requests for the current API reference sections to the new individual reference pages. There will not be a versioned or latest version of a single reference to link to.
  3. Make sure the Markdown API generator tool can be rebuilt, the build instructions are clear (both repos), the tool can consistently generate new API Markdown pages, and these generated pages can be added and build with the website.
sftim commented 3 years ago

Another option that we could consider:

  1. Replace all links (across localizations as well, if feasible) with a new shortcode. At this point the shortcode renders as a link to the old / existing API reference.
  2. Draft a revised shortcode that links to the new API reference.
  3. Update the “old” API generator so that the rendered API reference hyperlinks to the new one.
    • This doesn't have to be perfect.
  4. Publish the new API reference and switch the the updated shortcode.
  5. Update the old API reference to include links to the new API reference.
feloy commented 3 years ago

Should we consider the coexistence of the two API reference representations for one or two release cycles? It is a common practice for websites to serve two flavors of some contents till a complete switch over proves to be welcomed.

Yes, that seems reasonable. I have not thought about the timeline for completely switching over. I suggest to decide on a timeline and set up a date to communicate this decision to the community (how and when these changes will finally happen). There may be issues logged. I see a short list of things to do:

  1. Update the intra-site links to refer to the new API reference pages.

I'm updating the intra-links. I've changed the links in the Concepts and Tasks, I still have to change the links in the Reference section.

The changes are visibile in the PR https://github.com/kubernetes/website/pull/23294

  1. Determine how to redirect external requests for the current API reference sections to the new individual reference pages. There will not be a versioned or latest version of a single reference to link to.
  2. Make sure the Markdown API generator tool can be rebuilt, the build instructions are clear (both repos), the tool can consistently generate new API Markdown pages, and these generated pages can be added and build with the website.
sftim commented 3 years ago

PR #23294 has merged. I'll approach Kubernetes SIG Architecture to co-ordinate communications and other tasks about deprecating and moving away from the existing API documentation.

SIG Docs' preference is to shift away as part of a Kubernetes release (eg v1.21). One reason for this is that each release so far has broken the old API reference URLs anyway.

sftim commented 3 years ago

@reylejano are you happy to have this tracked against v1.21?

neolit123 commented 3 years ago

The existing API documentation (example uses a different style from the rest of the website, is harder to navigate than it could be, and is also difficult to localize.

one good thing about the old format is that it was easier to search the whole API reference using a browser Ctrl+F (search), since it was all on the same page.

sftim commented 3 years ago

Perhaps there's a way to improve the search(ability). For example, add an index to the foot of each API reference page.

liggitt commented 3 years ago

Thanks for the update! I especially like how much better it renders on mobile devices.

Here is some initial feedback on the organization and rendering:

  1. The primary organization of the API is by API group → version → resource/kind. From https://kubernetes.io/docs/reference/kubernetes-api/, it is hard to know where documentation for types in various group/versions are, and I had to guess about which category a given type was placed into. Can we make the API group+version explicit in the index/navigational structure or overview pages?
  2. The URLs include the categorizations (".../services-resources/...", ".../workloads-resources/...", etc) which are somewhat arbitrary. Making the URLs based on the API group/version/kind will produce more stable URLs (makes it easier to address item 7 later in a follow-up).
  3. The representation of kinds with multiple versions available is confusing/missing:
    • For example, https://kubernetes.io/docs/reference/kubernetes-api/workloads-resources/ lists "HorizontalPodAutoscaler" and "HorizontalPodAutoscaler v2beta2". The first is implicitly in the autoscaling/v1 API group/version, but that isn't clear. We need to make it very clear what version the documentation is for (someone following the v1 HorizontalPodAutoscaler docs will not successfully construct a v2beta2 HorizontalPodAutoscaler manifest)
    • Documentation for some beta types seems to be missing altogether (Ingress v1beta1, for example).
  4. Some pages mix top-level types and nested type definitions. For example, https://kubernetes.io/docs/reference/kubernetes-api/workloads-resources/ has Pod, ReplicaSet, Deployment, etc, which can be created as objects, but also has Container, which only exists nested inside top-level types. I expected only top-level types to be linked from the overview pages.
  5. Grouping the "common parameters" page by operation would be helpful (parameters for get, list, watch, create, update, patch, delete, deletecollection operations, etc). Even if this duplicated some parameters (like dryRun, fieldManager, etc), it would make it easier to understand which bucket of parameters could be used in a given request).
  6. It's unclear what causes a type to appear in https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/ ... these types I would have expected to have been on specific pages: DownwardAPIVolumeFile, ExecAction, HTTPGetAction, JSONSchemaProps, KeyToPath, NodeAffinity, NodeSelectorRequirement, PodAffinity, PodAntiAffinity, ResourceFieldSelector, TCPSocketAction
  7. Some of the type categorizations seemed odd to me... the lines between workload (pod, deployment, etc) + node (node, runtimeclass, etc) + scheduling (priorityclass, binding) are pretty blurry. I'd suggest getting sig-apps / sig-scheduling / sig-node involved to work on that organization. If we make the URLs not depend on the categorizations (see suggestion 2 above), it will be lower impact to switch up the categorizations in a follow-up.
reylejano commented 3 years ago

@reylejano are you happy to have this tracked against v1.21?

@sftim Yes, I'm comfortable with this being tracked against v1.21. I want to review and capture any process changes for the release for the docs role handbook timeline.

sftim commented 3 years ago

We also need to add a redirect so that https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/ points to the new page at https://kubernetes.io/docs/reference/labels-annotations-taints/ …as mentioned in https://github.com/kubernetes/website/issues/24661#issuecomment-765386803

kbhawkey commented 3 years ago

Hi. As part of the migration, I'd like to understand the API categories and work on resolving some of the questions about navigation.

Re: Comment:

The primary organization of the API is by API group → version → resource/kind. From https://kubernetes.io/docs/reference/kubernetes-api/, it is hard to know where documentation for types in various group/versions are, and I had to guess about which category a given type was placed into. Can we make the API group+version explicit in the index/navigational structure or overview pages?

Re: More comments:

The URLs include the categorizations (".../services-resources/...", ".../workloads-resources/...", etc) which are somewhat arbitrary. Making the URLs based on the API group/version/kind will produce more stable URLs (makes it easier to address item 7 later in a follow-up).
The representation of kinds with multiple versions available is confusing/missing:

    For example, https://kubernetes.io/docs/reference/kubernetes-api/workloads-resources/ lists "HorizontalPodAutoscaler" and "HorizontalPodAutoscaler v2beta2". The first is implicitly in the autoscaling/v1 API group/version, but that isn't clear. We need to make it very clear what version the documentation is for (someone following the v1 HorizontalPodAutoscaler docs will not successfully construct a v2beta2 HorizontalPodAutoscaler manifest)
    Documentation for some beta types seems to be missing altogether (Ingress v1beta1, for example).
sftim commented 3 years ago

I need to talk to SIG Architecture about stability (or not) of the URLs here.

neolit123 commented 3 years ago

The URLs include the categorizations (".../services-resources/...", ".../workloads-resources/...", etc) which are somewhat arbitrary. Making the URLs based on the API group/version/kind will produce more stable URLs (makes it easier to address item 7 later in a follow-up).

i think this is rather important. categorizations are a nice to have and can be implement with labels / filters ALA github.

sftim commented 3 years ago

There is a question about missing beta types. Log a separate issue?

I agree, that sounds like a separate issue.

sftim commented 3 years ago

/triage accepted

sftim commented 3 years ago

I checked in with SIG Architecture

Outcome from that call:

/assign

sftim commented 3 years ago

For comparison, here are two other Kubernetes-related API references:

kbhawkey commented 3 years ago

Catching up on this discussion. Can you provide more details on: We can separate out the “presentation” grouping of resources (eg workloads, policy, etc) from the URL of the page about a particular resource in the API - and we should do so that we have stable URIs for the API reference pages.

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 3 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/website/issues/25505#issuecomment-894944482): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
sftim commented 3 years ago

/reopen /lifecycle frozen

k8s-ci-robot commented 3 years ago

@sftim: Reopened this issue.

In response to [this](https://github.com/kubernetes/website/issues/25505#issuecomment-895048440): >/reopen >/lifecycle frozen Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
sftim commented 2 years ago

I'm happy to keep working on this as time allows /assign

sftim commented 1 year ago

/unassign

I haven't made much progress - when I get chunks of time, I'm spending those instead on other tasks; notably, https://kubernetes.io/docs/concepts/services-networking/service/

k8s-triage-robot commented 7 months ago

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

sftim commented 7 months ago

/triage accepted

Still useful to provide a consistent reference.

sftim commented 6 months ago

/wg api-expression

What I'd like to see: