Closed tkan145 closed 5 months ago
eguzki
commented
5 months ago resty-http was downgraded to 0.15 for ARM support https://github.com/3scale/APIcast/commit/6a09893a7277225542a040afd62c35f5c55e40ee
Can we test that this image can be built out of this PR using ARM arch host? If you do not have available, I can ask some other member of the team to test it.
tkan145
commented
5 months ago So I tried to build ARM image using docker from masterbranch
▲ APIcast make dev-build
/usr/bin/docker buildx build --platform linux/arm64 -t apicast-development:latest \
--build-arg OPENRESTY_RPM_VERSION=1.19.3 \
--build-arg LUAROCKS_VERSION=2.3.0 \
/3scale/APIcast -f Dockerfile.devel
=> [ 8/16] RUN yum config-manager --add-repo http://packages.dev.3sca.net/dev_packages_3sca_net.repo 3.7s
=> ERROR [ 9/16] RUN yum install -y openresty-1.19.3 openresty-resty-1.19.3 openresty-opentelemetry-1.19.3 openresty-opentracing-1.19.3 opentracing-cpp-devel-1.3.0 6.9s
------
> [ 9/16] RUN yum install -y openresty-1.19.3 openresty-resty-1.19.3 openresty-opentelemetry-1.19.3 openresty-opentracing-1.19.3 opentracing-cpp-devel-1.3.0 libopentraci
cpp1-1.3.0 jaegertracing-cpp-client-0.3.1-13.el8:
4.901 Devel packages from 3Scale 81 kB/s | 220 kB 00:02
6.661 Error:
6.661 Problem 1: conflicting requests
6.661 - package jaegertracing-cpp-client-0.3.1-13.el8.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by jaegertracing-cpp-client-0.3.1-13.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by jaegertracing-cpp-client-0.3.1-13.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libc.so.6(GLIBC_2.14)(64bit) needed by jaegertracing-cpp-client-0.3.1-13.el8.x86_64 from packages.dev.3sca.net
6.661 Problem 2: cannot install the best candidate for the job
6.661 - package openresty-1.19.3-23.el8.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides openresty-pcre >= 8.42-1 needed by openresty-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides openresty-zlib >= 1.2.11-3 needed by openresty-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 Problem 3: cannot install the best candidate for the job
6.661 - nothing provides openresty >= 1.19.3-23.el8 needed by openresty-resty-1.19.3-23.el8.noarch from packages.dev.3sca.net
6.661 Problem 4: cannot install the best candidate for the job
6.661 - package openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides libpthread.so.0(GLIBC_2.2.5)(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libpthread.so.0(GLIBC_2.3.2)(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides ld-linux-x86-64.so.2()(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libm.so.6(GLIBC_2.2.5)(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libpthread.so.0(GLIBC_2.12)(64bit) needed by openresty-opentelemetry-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 Problem 5: cannot install the best candidate for the job
6.661 - package openresty-opentracing-1.19.3-23.el8.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides libc.so.6(GLIBC_2.14)(64bit) needed by openresty-opentracing-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libopentracing.so.1()(64bit) needed by openresty-opentracing-1.19.3-23.el8.x86_64 from packages.dev.3sca.net
6.661 Problem 6: cannot install the best candidate for the job
6.661 - package opentracing-cpp-devel-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides libopentracing.so.1()(64bit) needed by opentracing-cpp-devel-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libopentracing_mocktracer.so.1()(64bit) needed by opentracing-cpp-devel-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libopentracing-cpp1 = 1.3.0-26.el8arches needed by opentracing-cpp-devel-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 Problem 7: cannot install the best candidate for the job
6.661 - package libopentracing-cpp1-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net does not have a compatible architecture
6.661 - nothing provides libc.so.6(GLIBC_2.14)(64bit) needed by libopentracing-cpp1-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 - nothing provides ld-linux-x86-64.so.2()(64bit) needed by libopentracing-cpp1-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 - nothing provides ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) needed by libopentracing-cpp1-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.661 - nothing provides libdl.so.2(GLIBC_2.2.5)(64bit) needed by libopentracing-cpp1-1.3.0-26.el8arches.x86_64 from packages.dev.3sca.net
6.662 (try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
------
Dockerfile.devel:30 Checking http://packages.dev.3sca.net/ I don't see any aarch64 packages. Perhaps you have a better way to build the image?
guicassolato
commented
5 months ago I’ve tried this but the verification steps won’t work OOTB on my system, due to missing RPMs for arm64 (aarch64). The packages we need (i.e. OpenResty 1.19.3-x and related ones within the same “family”, e.g. OpenTelemetry, OpenTracing, etc) are not available for this arch, neither in the default repos, nor in http://packages.dev.3sca.net/.
Searching my notes after #1381, I found out a few interesting things. Starting with the fact that, back then, I only touched Dockerfile.devel, but never Dockerfile. This means I was able to build the devel/ci container image on darwin/arm64, but never the runtime image, which is the one we use to build for prod IIRC.
Another important piece is that apparently I failed then to build the devel/ci image while targeting linux/arm64 platform. What I have succeeded doing was building on darwin/arm64 for linux/amd64.
To not completely diminish that as 100% useless, I recon it may have helped people to boot up the dev env container and run the test suite on a MacOS with Mx chip. But, in the end, we were just working around the limitations for running an arm64 image on an arm64 platform for dev purposes, but that’s all.
Back to this PR... While still unable to make runtime-image, there was only so much I could do. Here’s a summary:
make dev-build IMAGE=quay.io/3scale/apicast-ci:openresty-1.19.3-pr1434-amd64
make development IMAGE=quay.io/3scale/apicast-ci:openresty-1.19.3-pr1434-amd64Then, inside the development container:
make dependencies
make busted
make proveResult: SUCCESS
make runtime-build # <========= FAILED
cd dev-environments/https-proxy-upstream-tlsv1.3
make certs
make gateway
curl --resolve get.example.com:8080:127.0.0.1 -v "http://get.example.com:8080/?user_key=123"
docker compose -p https-proxy-upstream-tlsv13 logs -f proxyResult: FAILED
Inside the development container:
THREESCALE_DEPLOYMENT_ENV=staging APICAST_LOG_LEVEL=debug APICAST_WORKER=1 APICAST_CONFIGURATION_LOADER=lazy APICAST_CONFIGURATION_CACHE=0 THREESCALE_PORTAL_ENDPOINT=https://token@3scale-admin.example.com ./bin/apicastFrom the host:
APICAST_IP=$(docker inspect apicast_build_0-development-1 | yq e -P '.[0].NetworkSettings.Networks.apicast_build_0_default.IPAddress' -)
curl -i -k -H "Host: example.com:443" -H "Accept: application/json" -H "Authorization: Bearer ${ACCESS_TOKEN}" "http://${APICAST_IP}:8080/foo"Output:
HTTP/1.1 404 Not Found
Server: openresty
Date: Thu, 01 Feb 2024 10:56:20 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-aliveResult: SUCCESS (?)
tkan145
commented
5 months ago Thanks @guicassolato, for the last test, did you replace the placeholder values with actual 3scale values? It may also fail if you are using an older version of lua-resty-http (0.15), maybe delete the lua_modules folder and rerun make dependencies?
@eguzki so I guess we are good to merge this one?
eguzki
commented
5 months ago building on darwin/arm64 for linux/amd64
Thanks @guicassolato
That was what I was missing. I was wondering how the hell you build dev image. building on darwin/arm64 for linux/amd64 answers that.
The aim is to allow arm64 based users to develop APIcast. If they cannot build runtime image for arm64, that's unfortunate but not a blocking issue if they can still build and run amd64 images.
eguzki
commented
5 months ago There is a failing check about codecov. It says that the tests only cover the 72% of the added code in this PR (not the overall code coverage). While we could be more restrictive about this, I tend to think that 70% of code coverage either for the patch or the project is good enough and should not be blocking the merge.
tkan145
commented
5 months ago codecov is a strange one. It's fine now :sweat_smile:
tkan145
commented
5 months ago Rebased!
eguzki
commented
5 months ago @tkan145 the second part of the verification steps cannot be run. You did not specify the 3scale configuration fetched from 3scale API, hence I cannot reproduce.
Nevertheless, I consider the first part good enough. And together with the (old) e2e tests passing, I consider the PR tested.
tkan145
commented
5 months ago Updated verification steps.
Because this has been approved. I will merge now
What
Fix https://issues.redhat.com/browse/THREESCALE-10278
lua-resty-http 0.17.1 is also required for https://issues.redhat.com/browse/THREESCALE-5105
This PR is mainly a refactoring of existing code so no additional integration tests/unittests are added.
Verification Steps
Request should return 200
Run APIcast locally
Run query
curl -i -k -H "Host: default-product.staging.example.com:443" "http://${APICAST_IP}:8080/?user_key="
HTTP/1.1 200 OK
Server: openresty
Date: Tue, 30 Jan 2024 05:08:26 GMT Content-Type: application/json
Content-Length: 702
Connection: keep-alive
x-3scale-echo-api: echo-api/1.0.3
vary: Origin
x-content-type-options: nosniff
x-envoy-upstream-service-time: 0
2024/02/14 01:42:12 [debug] 763706#763706: *2 remote_v2.lua:268: proxy_configs_per_page(): proxy configs get status: 200 url: https://user_key@3scale-admin.example.con/admin/api/account/proxy_configs/production.json?host=default-product.staging.example.com&page=1&per_page=500&version=latest body: {"proxy_configs": .... }
{ "services": [ { "proxy": { "error_headers_auth_failed": "text\/plain; charset=us-ascii", "error_headers_limits_exceeded": "text\/plain; charset=us-ascii", "error_headers_auth_missing": "text\/plain; charset=us-ascii", "error_headers_no_match": "text\/plain; charset=us-ascii", "error_status_no_match": 404, "error_status_auth_failed": 403, "error_status_limits_exceeded": 429, "error_status_auth_missing": 403, "secret_token": "Shared_secret_sent_from_proxy_to_API_backend_09df7e84d9ba36d8", "hostname_rewrite": null, "oidc_issuer_endpoint": null, "jwt_claim_with_client_id": null, "jwt_claim_with_client_id_type": null, "auth_user_key": "user_key", "auth_app_id": "app_id", "auth_app_key": "app_key", "oauth_login_url": null, "proxy_rules": [ { "http_method": "GET", "pattern": "\/", "delta": 1, "redirect_url": null, "querystring_parameters": {}, "position": 1, "parameters": {}, "metric_system_name": "hits", "last": false, "owner_type": "Proxy" ... } ], "error_auth_missing": "Authentication parameters missing", "api_test_path": "\/", "api_test_success": null, "apicast_configuration_driven": true, "oidc_issuer_type": "keycloak", "staging_domain": "default-product.staging.example.com", "production_domain": "default-product.production.example.com", "endpoint": "https:\/\/default-product.production.example.com: 443", "error_limits_exceeded": "Usage limit exceeded", "deployed_at": null, "backend": { "endpoint": "https:\/\/su1.3scale.net", "host": "su1.3scale.net" }, "error_no_match": "No Mapping Rule matched", "valid?": true, "service_backend_version": "1", "hosts": [ "default-product.production.example.com", "default-product.staging.example.com" ], "error_auth_failed": "Authentication failed", "lock_version": 1, "policy_chain": [ { "name": "apicast", "configuration": {}, "version": "builtin" } ], "endpoint_port": 443, "sandbox_endpoint": "https:\/\/default-product.staging.example.com: 443", "authentication_method": "1", "api_backend": "https:\/\/echo-api.3scale.net: 443", "credentials_location": "query", "hostname_rewrite_for_sandbox": "echo-api.3scale.net" },
"backend_authentication_type": "service_token", "description": "", "name": "Default Product", "backend_version": "1", "proxiable?": true, "system_name": "default_product", ... }, ] }