search

kubernetes-sigs / apiserver-runtime

Libraries for implementing aggregated apiservers
Apache License 2.0
84 stars 59 forks source link

Anyway to avoid etcd dependency when it's not needed? #44

Closed fengye87 closed 2 years ago

fengye87 commented 3 years ago

I'm building an apiserver with this project which doesn't use etcd as backend. However, the resulting binary still requires a --etcd-servers flag to get started normally, and it would keep connecting to that address until a real etcd service is connectd. And during this connecting period, the api endpoint would keep returning 503 error.

So, is there anyway I can avoid the etcd dep? If not, any suggestion to how to make it possible? I may compose a PR here.

yue9944882 commented 3 years ago

@fengye87 i think it's a duplicate of https://github.com/kubernetes-sigs/apiserver-builder-alpha/issues/583. and i will get back w/ feasible fixes after reproducing that locally.

yue9944882 commented 3 years ago

https://github.com/kubernetes-sigs/apiserver-builder-alpha/issues/583#issuecomment-818620597 @fengye87 i saw a few warning level loggings from the apiserver when it's actually working w/o etcd, but the custom resources will be working fine. the e2e test https://github.com/kubernetes-sigs/apiserver-builder-alpha/pull/587 also confirms that..

yue9944882 commented 3 years ago

@fengye87 i cut a new release of apiserver-runtime in which etcd dependency can be completely removed by having WithoutEtcd() in the builder flow. can you try?

fengye87 commented 3 years ago

@yue9944882 I can confirm WithoutEtcd() works as expected, but doesn't solve my problem entirely.

I've created a sample to demonstrate my requirements. A few key points here:

In the previous commit of the sample, I can get it work but with an Etcd. In the current commit, I can remove the Etcd dependency only if I add one of CRUD operations to Foo, which would effectively disable the discovery of subresource Bar due to https://github.com/kubernetes-sigs/apiserver-runtime/blob/main/pkg/builder/builder_resource.go#L73. Moreover, if I do not add any CRUD operations to Foo, I have to provide a RESTOptionsGetter to the builder flow to avoid some nil pointers while starting the server. However, I still get a etcdclient: no available endpoints error with a proper RESTOptionsGetter.

yue9944882 commented 3 years ago

A dummy resource Foo, ideally without any CRUD endpoints

@fengye87 for Foo resource, just implementing resource.Object interface should be fine which will makes it a classic kubernetes resource. i'm not sure if all CRUD endpoints can be disabled at all but it doesn't seem to be a blocker for your case, is it?

A subresource Bar, which answers only to the Connect verb and has custom implementation

in the latest release of apiserver-runtime (v1.0.1), you can implement resource.ConnectorSubResource interface for your Bar definitions. Meanwhile, you're supposed to setup the connection between Foo and Bar as is shown in this example. basically i think your case is really close to the example project pod-exec from our peer project.

fengye87 commented 3 years ago

@yue9944882 I've just had a look at the example project pod-exec you mentioned, it is very close to my sample. Can the pod-exec apiserver run without etcd? In my sample project, as soon as I introduced the WithoutEtcd option, my apiserver just refuse to start up.

k8s-triage-robot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

k8s-triage-robot commented 3 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/apiserver-runtime/issues/44#issuecomment-927132580): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.