Add e2e test cases for plugins

tokers commented 3 years ago

Issue description

We don't cover plugins in e2e test suites, although we support them already. Here I want to initiate a plan to add test cases for most of all plugins in Apache APISIX.

[ ] batch-requests
[x] serverless
[x] redirect
[x] echo
[x] server-info
[x] response-rewrite
[x] proxy-rewrite
[ ] grpc-transcode
[x] fault-injection
[x] key-auth
[ ] jwt-auth
[x] basic-auth
[ ] authz-keycloak
[x] wolf-rbac
[ ] openid-connect
[x] hmac-auth
[x] cors
[x] uri-blocker
[x] ip-restriction
[x] referer-restriction
[x] consumer-restriction
[ ] limit-req
[ ] limit-conn
[x] limit-count
[ ] proxy-cache
[x] request-validation
[ ] proxy-mirror
[x] api-breaker
[x] traffic-split, supported by ApisixRoute in first-class.
[x] request-id
[x] prometheus
[ ] zipkin
[ ] skywalking
[ ] node-status
[ ] http-logger
[ ] tcp-logger
[ ] kafka-logger
[ ] udp-logger
[ ] syslog
[ ] log-rotate
[ ] error-log-logger
[ ] sls-logger
[ ] dubbo-proxy
[ ] mqtt-proxy
[ ] ua-restriction

If you're interested in writing some test cases for several plugins, please writing comments like:

I'd like to write e2e test cases for plugin "fault-injection".

Then, I'll create a dedicated issue and assign it to you. After the PR is merged, the checkbox in this description will be clicked.

Test case samples are here:

You may have to learn some usages about ginkgo.

A test case should be a complete e2e case, it contains the configuration create/update/delete on "control plane" (in our case is applying k8s resources like ApisixRoute, ApisixUpstream to Kubernetes), then waiting for it to be effective, and sending requests to "data plane" (APISIX), asserting the response is valid.

When you want to run these cases on your local environment, be sure a Kubernetes cluster is installed, you can create a kubernetes cluster with minikube or kind.

You may try to push images in advance or your cases might be timed out when first run. Related images are:

apache/apisix-ingress-controller:dev
apache/apisix:dev
bitnami/etcd:3.4.14-debian-10-r0
kennethreitz/httpbin

You can change the Describe or It block in your case to FDescribe or FIt so only these cases will be run when you executing make e2e-test directive. By default make e2e-test will build apisix-ingress-controller from scratch, you can add E2E_SKIP_BUILD=1 to skip this step, and using E2E_CONCURRENCY to control the concurrency of cases.

gxthrj commented 3 years ago

Great Job ~

gxthrj commented 3 years ago

Great Job ~

Firstsawyou commented 3 years ago

I'd like to write e2e test cases for plugin "key-auth".

juzhiyuan commented 3 years ago

I'd like to write e2e test cases for plugin "key-auth".

any update?

Firstsawyou commented 3 years ago

I'd like to write e2e test cases for plugin "key-auth".

any update?

Currently, there is no configuration that supports the consumer object. I need to wait for the consumer configuration to be supported before submitting this test.

Firstsawyou commented 3 years ago

I'd like to write e2e test cases for plugin "referer-restriction".

fhuzero commented 3 years ago

I'd like to write e2e test cases for plugin "server-info".

fhuzero commented 3 years ago

Help needed: is there any way to run test cases for just one plugin? I want to run test cases I wrote for plugin "server-info". Modifying ginkgo command under PHONY e2e-test in Makefile seems to have no effect, ginkgo still runs all test cases.

tokers commented 3 years ago

Help needed: is there any way to run test cases for just one plugin? I want to run test cases I wrote for plugin "server-info". Modifying ginkgo command under PHONY e2e-test in Makefile seems to have no effect, ginkgo still runs all test cases.

Yep, please read the issue description.

fhuzero commented 3 years ago

Thanks for your reply!

Help needed: is there any way to run test cases for just one plugin? I want to run test cases I wrote for plugin "server-info". Modifying ginkgo command under PHONY e2e-test in Makefile seems to have no effect, ginkgo still runs all test cases.

Yep, please read the issue description.

Thanks! It's done by changing Describe to FDescribe. Sorry for not examining the issue description carefully. Just went through a hard time configuring the environment.

tokers commented 3 years ago

Thanks for your reply!

Help needed: is there any way to run test cases for just one plugin? I want to run test cases I wrote for plugin "server-info". Modifying ginkgo command under PHONY e2e-test in Makefile seems to have no effect, ginkgo still runs all test cases.

Yep, please read the issue description.

Thanks! It's done by changing Describe to FDescribe. Sorry for not examining the issue description carefully. Just went through a hard time configuring the environment.

Feel free to ask any questions here.

fhuzero commented 3 years ago

Now I have a workable version for "server-info" test cases. But as for implementation details and how the test is done I would like to raise some questions for discussion. Shall we create a dedicated issue for "server-info" plugin test cases or I just start a PR so we can discuss in that PR?

tokers commented 3 years ago

Now I have a workable version for "server-info" test cases. But as for implementation details and how the test is done I would like to

raise some questions for discussion. Shall we create a dedicated issue for "server-info" plugin test cases or I just start a PR so we can discuss in that PR?

I think so, you can go ahead and create it.

tokers commented 3 years ago

Now I have a workable version for "server-info" test cases. But as for implementation details and how the test is done I would like to raise some questions for discussion. Shall we create a dedicated issue for "server-info" plugin test cases or I just start a PR so we can discuss in that PR?

Merged #406 , thanks!