automation and filedescriptorset usage

[clux]

minor tweaks, i'm still struggling to get the servicegenerator to do something, but saw this pattern around and thought, this is probably a good way to sanity assert. had misunderstood servicegenerator (same as you, and had failed to remember). will explore the FDS path.

• moves generated protos list output to protos.list (with include_str! -> split)
• moves the fds output to protos.fds
• using both files as inputs in build.rs to avoid copying anything
• adds a POC file appending mechanics at the end of build.rs (committed a comment line for each api)

[clux]

goofed around a bit with just in here as well to try to order all the scripts in here accordingly:

[clux]

Edited a lot and updated. Went with your FileDescriptorSet route, which seems viable. I think we can continue on from something like this and find corresponding types from the api-resources.json and inject as appropriate.

I'm going to be away for a few days. If you want this feel free to squash (to get rid of my failed ServiceGenerator experiments), or fixup. Otherwise I can continue on later in the week (but probably not super actively).

[kazk]

https://github.com/influxdata/pbjson (pbjson-build) generates serde::Serialize/serde::Deserialize implementations from FileDescriptorSet. Maybe we can learn something from them.

[kazk]

I got spec, status, and condition types for #4 easily, so I'm pushing it here. Hope you didn't have any local changes.

[kazk]

Can we open a new PR without FileDescriptorSet for now?

With the new transform.jq, Pod looks like this:

{
  "api.core.v1.Pod": {
    "name": "pods",
    "namespaced": true,
    "apiGroupVersion": "v1",
    "group": "",
    "version": "v1",
    "kind": "Pod",
    "proto": "api.core.v1.Pod",
    "rust": "api::core::v1::Pod",
    "metadata": "apimachinery::pkg::apis::meta::v1::ObjectMeta",
    "spec": "api::core::v1::PodSpec",
    "status": "api::core::v1::PodStatus",
    "condition": "api::core::v1::PodCondition",
    "scopedVerbs": {
      "all": [
        "list"
      ],
      "namespaced": [
        "create",
        "delete",
        "deletecollection",
        "get",
        "list",
        "patch",
        "update"
      ]
    },
    "paths": [
      "/api/v1/pods",
      "/api/v1/namespaces/{namespace}/pods",
      "/api/v1/namespaces/{namespace}/pods/{name}"
    ],
    "subresources": [
      {
        "name": "attach",
        "scopedVerbs": {
          "namespaced": [
            "connect"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/attach"
        ]
      },
      {
        "name": "binding",
        "scopedVerbs": {
          "namespaced": [
            "create"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/binding"
        ]
      },
      {
        "name": "ephemeralcontainers",
        "scopedVerbs": {
          "namespaced": [
            "get",
            "patch",
            "update"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/ephemeralcontainers"
        ]
      },
      {
        "name": "eviction",
        "scopedVerbs": {
          "namespaced": [
            "create"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/eviction"
        ]
      },
      {
        "name": "exec",
        "scopedVerbs": {
          "namespaced": [
            "connect"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/exec"
        ]
      },
      {
        "name": "log",
        "scopedVerbs": {
          "namespaced": [
            "get"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/log"
        ]
      },
      {
        "name": "portforward",
        "scopedVerbs": {
          "namespaced": [
            "connect"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/portforward"
        ]
      },
      {
        "name": "proxy",
        "scopedVerbs": {
          "namespaced": [
            "connect"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/proxy",
          "/api/v1/namespaces/{namespace}/pods/{name}/proxy/{path}"
        ]
      },
      {
        "name": "status",
        "scopedVerbs": {
          "namespaced": [
            "get",
            "patch",
            "update"
          ]
        },
        "paths": [
          "/api/v1/namespaces/{namespace}/pods/{name}/status"
        ]
      }
    ]
  }
}

• GVK, apiVersion
• Name
• metadata type for Metadata trait
• spec type for HasSpec trait
• status type for HasStatus trait
• condition type for HasConditions trait
• namespaced is true if resource can be namespaced. It can still have all verbs like list.
• Scoped resource verbs
• Scoped subresource Verbs

[clux]

That's definitely a much better json to work with. and have also been sitting on this PR for too long :(

The FileDescriptorSet is only there so we can use the generated json to create impls, and atm it doesn't really do a whole lot; traverses the json path (which is probably wrong now), and appends a bit to the files. Are you suggesting we just comment out all of the FDS stuff from 39 -> 83 for now? Because that makes sense to me.

[clux]

Guess there's also the rust structs i wrote for it in lib.rs, but we can drop all of that.

[kazk]

traverses the json path (which is probably wrong now), and appends a bit to the files.

I added a separate JSON transformed.json, so it should still work. We can remove the old one in the new PR.

Are you suggesting we just comment out all of the FDS stuff from 39 -> 83 for now? Because that makes sense to me.

I think we should create a new PR with automation and restructuring of the project (a separate codegen crate) that can be merged. Using FDS with transformed JSON is necessary, but it needs much more work, and it's difficult to iterate/collaborate without having the new structure merged first.

[kazk]

Or, we can just comment out the code and merge this after cleaning up some commits and fixing DCO.

[clux]

I can remove the FDS stuff from this PR quickly if that's ok, then maybe just merge this and I'll open a new PR later with my stashed FDS stuff.

[clux]

This commit builds and removes my changes to the generated files (it just maintains the automation): https://github.com/kube-rs/k8s-pb/commit/64ebaa4f113b1c78fcc27cc7054ab6447a765f81 if it's ok you can git cherry-pick it in (wasn't able to create a pr into this branch).

[kazk]

(wasn't able to create a pr into this branch)

Yeah, that's because this branch is in your fork.

I'll open a new PR later with my stashed FDS stuff.

Sorry if the changes I pushed I messed it up. This is also why I'd like to merge this, so we can open PRs branching from here. I'll cherry pick. Is it ok if I just drop the commits for ServiceGenerator attempts (squash them)?

[clux]

Nah, it's fine. I didn't want to impose by pushing straight into this.

But, yeah sounds great. Cherry pick, and rebase drop / squash away what you don't like! Will be great to have this stuff in main!

[kazk]

Superseded by #5. Pushed to a new branch just in case I messed up something while rebasing.