Closed par97 closed 3 months ago
I have verified this PR works in my test env:
(ramen) fusion@s194:~/keslerzhu/ramen/e2e$ go test -kubeconfig-hub ./kubeconfig/hub/kubeconfig -kubeconfig-c1 ./kubeconfig/c1/kubeconfig -kubeconfig-c2 ./kubeconfig/c2/kubeconfig
2024-05-06T15:28:41.441+0800 INFO e2e/main_test.go:57 TestSuites
2024-05-06T15:28:41.441+0800 INFO deployers/subscription.go:30 enter Deploy Deployment/Subscription
2024-05-06T15:28:41.538+0800 INFO dractions/actions.go:28 enter EnableProtection Deployment/Subscription
2024-05-06T15:28:41.541+0800 INFO dractions/retry.go:40 could not get placement decision, retry in 30 seconds
2024-05-06T15:29:11.546+0800 INFO dractions/retry.go:29 got placementdecision name subscription-busybox-decision-1
2024-05-06T15:29:11.546+0800 INFO dractions/actions.go:55 get placementdecision subscription-busybox-decision-1
2024-05-06T15:29:11.549+0800 INFO dractions/actions.go:63 placementdecision clusterName: rdr-dr1
2024-05-06T15:29:11.549+0800 INFO dractions/actions.go:74 update placement subscription-busybox annotation
2024-05-06T15:29:11.558+0800 INFO dractions/actions.go:80 create drpc subscription-busybox
2024-05-06T15:29:11.585+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:29:41.613+0800 INFO dractions/retry.go:62 drpc subscription-busybox LastGroupSyncTime is nil
2024-05-06T15:29:41.613+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:30:11.645+0800 INFO dractions/retry.go:62 drpc subscription-busybox LastGroupSyncTime is nil
2024-05-06T15:30:11.645+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:30:41.663+0800 INFO dractions/retry.go:56 drpc subscription-busybox is ready
2024-05-06T15:30:41.663+0800 INFO dractions/actions.go:132 enter DRActions Failover
2024-05-06T15:30:41.667+0800 INFO dractions/retry.go:56 drpc subscription-busybox is ready
2024-05-06T15:30:41.667+0800 INFO dractions/actions.go:151 get drpc subscription-busybox
2024-05-06T15:30:41.670+0800 INFO dractions/actions.go:158 get drpolicy dr-policy
2024-05-06T15:30:41.675+0800 INFO dractions/retry.go:29 got placementdecision name subscription-busybox-decision-1
2024-05-06T15:30:41.676+0800 INFO dractions/retry.go:136 get placementdecision subscription-busybox-decision-1
2024-05-06T15:30:41.678+0800 INFO dractions/retry.go:144 placementdecision clusterName: rdr-dr1
2024-05-06T15:30:41.678+0800 INFO dractions/actions.go:170 failover to cluster: rdr-dr2
2024-05-06T15:30:41.678+0800 INFO dractions/actions.go:175 update drpc subscription-busybox
2024-05-06T15:30:46.700+0800 INFO dractions/retry.go:124 current drpc subscription-busybox phase is FailingOver, expecting FailedOver, retry in 30 seconds
2024-05-06T15:31:16.704+0800 INFO dractions/retry.go:114 drpc subscription-busybox phase is FailedOver
2024-05-06T15:31:16.707+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:31:16.707+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:31:46.733+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:31:46.733+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:32:16.740+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:32:16.740+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:32:46.770+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:32:46.770+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:33:16.776+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:33:16.776+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:33:46.804+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:33:46.804+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:34:16.818+0800 INFO dractions/retry.go:56 drpc subscription-busybox is ready
2024-05-06T15:34:16.818+0800 INFO dractions/actions.go:189 enter DRActions Relocate
2024-05-06T15:34:16.823+0800 INFO dractions/retry.go:56 drpc subscription-busybox is ready
2024-05-06T15:34:16.823+0800 INFO dractions/actions.go:208 get drpc subscription-busybox
2024-05-06T15:34:16.826+0800 INFO dractions/actions.go:215 get drpolicy dr-policy
2024-05-06T15:34:16.832+0800 INFO dractions/retry.go:29 got placementdecision name subscription-busybox-decision-1
2024-05-06T15:34:16.832+0800 INFO dractions/retry.go:136 get placementdecision subscription-busybox-decision-1
2024-05-06T15:34:16.835+0800 INFO dractions/retry.go:144 placementdecision clusterName: rdr-dr2
2024-05-06T15:34:16.835+0800 INFO dractions/actions.go:227 relocate to cluster: rdr-dr1
2024-05-06T15:34:16.835+0800 INFO dractions/actions.go:232 update drpc subscription-busybox
2024-05-06T15:34:21.856+0800 INFO dractions/retry.go:124 current drpc subscription-busybox phase is Initiating, expecting Relocated, retry in 30 seconds
2024-05-06T15:34:51.864+0800 INFO dractions/retry.go:124 current drpc subscription-busybox phase is Relocating, expecting Relocated, retry in 30 seconds
2024-05-06T15:35:21.868+0800 INFO dractions/retry.go:124 current drpc subscription-busybox phase is Relocating, expecting Relocated, retry in 30 seconds
2024-05-06T15:35:51.896+0800 INFO dractions/retry.go:124 current drpc subscription-busybox phase is Relocating, expecting Relocated, retry in 30 seconds
2024-05-06T15:36:21.901+0800 INFO dractions/retry.go:114 drpc subscription-busybox phase is Relocated
2024-05-06T15:36:21.905+0800 INFO dractions/retry.go:91 drpc subscription-busybox condition PeerReady is not True
2024-05-06T15:36:21.905+0800 INFO dractions/retry.go:69 drpc subscription-busybox is not ready yet, retry in 30 seconds
2024-05-06T15:36:51.931+0800 INFO dractions/retry.go:56 drpc subscription-busybox is ready
2024-05-06T15:36:51.931+0800 INFO dractions/actions.go:93 enter DRActions DisableProtection
2024-05-06T15:36:51.931+0800 INFO dractions/actions.go:112 delete drpc subscription-busybox
2024-05-06T15:36:51.947+0800 INFO dractions/actions.go:118 get placement subscription-busybox
2024-05-06T15:36:51.950+0800 INFO dractions/actions.go:126 updated placement subscription-busybox annotation
2024-05-06T15:36:51.966+0800 INFO deployers/subscription.go:61 enter Undeploy Deployment/Subscription
PASS
ok github.com/ramendr/ramen/e2e 490.696s
(ramen) fusion@s194:~/keslerzhu/ramen/e2e$
@par97 for this PR to work in the e2e environment, we need to create a Channel. As the script sets up the RDR env, and deploys and configures Ramen but expects the next tool to create the Channel. (see failure here: https://github.com/RamenDR/ramen/pull/1373 )
In the case of e2e we would need another option, or just take in a .yaml file with the options for the git link to the Channel sources and create a channel from the code to begin with. Example YAML can look like: https://github.com/RamenDR/ramen/blob/main/test/configs/deployment-k8s-regional-rbd.yaml
@ShyamsundarR The channel is a prereq to run the subscription + deployment workload. As documented in https://github.com/RamenDR/ramen/blob/main/docs/testing.md, we need first add a channel:
Ramen basic test use the [ocm-ramen-samples repo](https://github.com/RamenDR/ocm-ramen-samples). Before running the tests, you need to deploy a channel pointing this repo:
kubectl apply -k https://github.com/RamenDR/ocm-ramen-samples.git/channel?ref=main --context hub
I have plan to add check for the channel in the Validate Suite.
To add such channel, we could have a few options:
@ShyamsundarR The channel is a prereq to run the subscription + deployment workload. As documented in https://github.com/RamenDR/ramen/blob/main/docs/testing.md, we need first add a channel:
Ramen basic test use the [ocm-ramen-samples repo](https://github.com/RamenDR/ocm-ramen-samples). Before running the tests, you need to deploy a channel pointing this repo: kubectl apply -k https://github.com/RamenDR/ocm-ramen-samples.git/channel?ref=main --context hub
I have plan to add check for the channel in the Validate Suite.
To add such channel, we could have a few options:
1. still doc this step as prereq. 2. execute above kubectl apply command in the e2e-rdr.sh, or directly kubectl apply the saved local yaml file. 3. create these resources (namespace, channel) in go code.
I am suggesting (3) because in local setups or when experimenting with non-standard channels (forms of ocm-ramen-sample) it is easier to pass in the git source and create a channel for e2e to use.
I added this issue to track some review comments that we can fix subsequently, so that we can make progress on using what is available (most will potentially be resolved once we get to the Sub+CephFS and/or AppSet+Workload as we generalize the framework and utils functions)
@par97 other than this one comment I think we are ready to merge this in, once you clarify I can merge this PR based on the clarification.
This PR is to enable subscription application with busybox deployment for e2e test.