search

apache / apisix

The Cloud-Native API Gateway
https://apisix.apache.org/blog/
Apache License 2.0
14.39k stars 2.5k forks source link

help request: router dispatch not working as expected #7449

Closed 1renwuruiming closed 1 year ago

1renwuruiming commented 2 years ago

Description

nginx存在多种路由的匹配规则，apisix是否存在同样的方式，如果存在，是如何使用的呢

Environment

1renwuruiming commented 2 years ago

@juzhiyuan 我发送了一些邮件到您的邮箱，并提供了一些日志截图，想要咨询一下apisix对于复杂路由的匹配规则是怎样的，是否和nginx一样，又是如何使用的呢

juzhiyuan commented 2 years ago

Hi, please read the following documents:

  1. https://apisix.apache.org/docs/apisix/terminology/router/#configuration
  2. https://apisix.apache.org/docs/apisix/router-radixtree/
  3. https://github.com/api7/lua-resty-radixtree
juzhiyuan commented 2 years ago

Also, could you please paste the screenshot and descriptions into this issue? Then more team members could check. 🙏

NOTE: Don't forget to erase sensitive content.

1renwuruiming commented 2 years ago

我降低版本到2.13.1就没有这样的问题了，所以是否是最近几个版本更改了路由的匹配规则导致的呢？看到其他的issue中也有提到这样的现象

juzhiyuan commented 2 years ago

看到其他的issue中也有提到这样的现象

Could you please paste those issues here? Then I will cc maintainers to check the difference.

tzssangglass commented 2 years ago

Please provide the minimum reproducible steps

1renwuruiming commented 2 years ago

开会的时候描述过过程了，我不知道还可以怎么描述，是否检查一下版本更新的diff内容更好呢？对比一下2.13.1和之后的版本的路由这边的代码差别，现象是如果只有一个路由，不会出现404的情况，如果出现多个路由，就会无法路由到正确的环境地址

tzssangglass commented 2 years ago

开会的时候描述过过程了，我不知道还可以怎么描述，是否检查一下版本更新的diff内容更好呢？对比一下2.13.1和之后的版本的路由这边的代码差别，现象是如果只有一个路由，不会出现404的情况，如果出现多个路由，就会无法路由到正确的环境地址

ok, just give me a route for the problem, I will try it on 2.13.1 and the latest version.

1renwuruiming commented 2 years ago

我认为这是一个普遍的问题，当你拥有多个请求前缀相似的路由的情况下，新版本的apisix不能很好的转发，你可以模拟这样的场景来进行验证

tzssangglass commented 2 years ago

我认为这是一个普遍的问题，当你拥有多个请求前缀相似的路由的情况下，新版本的apisix不能很好的转发，你可以模拟这样的场景来进行验证

we have some test cases to cover, see: https://github.com/apache/apisix/blob/master/t/router/radixtree-uri-sanity.t#L256-L306

So I need you to give the exact replication scenario, otherwise I won't verify it blindly.

juzhiyuan commented 2 years ago

Also, could you please paste the screenshot and descriptions into this issue? Then more team members could check. 🙏

NOTE: Don't forget to erase sensitive content.

@1renwuruiming Again, because your screenshot has some sensitive information, so I can't paste them here instead of you directly.

Also, all people here are volunteers who love this community, and if you can't provide enough information, it's hard to help you resolve this issue.

1renwuruiming commented 2 years ago

@juzhiyuan 我需要将后台服务的地址提供给您吗？

1renwuruiming commented 2 years ago

我以什么样的方式来提供route比较合适呢？

tzssangglass commented 2 years ago

我以什么样的方式来提供route比较合适呢？

The form of route stored in etcd, text form, desensitized.

And the verification process.

1renwuruiming commented 2 years ago

好的，我整理一下给您

1renwuruiming commented 2 years ago

@tzssangglass 如何从etcd导出数据呢？我尝试了多种方式仍旧没有成功导出

tzssangglass commented 2 years ago

@tzssangglass 如何从etcd导出数据呢？我尝试了多种方式仍旧没有成功导出

you can use etcdctl, e.g. etcdctl get /apisix/routes/1 or etcdctl get / --prefix

1renwuruiming commented 2 years ago

[{ "key": "\/apisix\/consumers\/", "value": "init_dir" }, { "key": "\/apisix\/global_rules\/", "value": "init_dir" }, { "key": "\/apisix\/global_rules\/1", "value": "{\"id\":\"1\",\"create_time\":1656912914,\"update_time\":1657849534,\"plugins\":{\"api-breaker\":{\"break_response_code\":507,\"disable\":false,\"healthy\":{\"http_statuses\":[200],\"successes\":2},\"max_breaker_sec\":180,\"unhealthy\":{\"failures\":1,\"http_statuses\":[503,500]}},\"limit-req\":{\"burst\":2,\"disable\":false,\"key\":\"$x_forward_for $http_user_agent $uri $upstream_uri\",\"key_type\":\"var_combination\",\"nodelay\":true,\"rate\":3,\"rejected_code\":503,\"rejected_msg\":\"操作太快了，请稍后重试\"}}}" }, { "key": "\/apisix\/plugin_configs\/", "value": "init_dir" }, { "key": "\/apisix\/plugin_metadata\/", "value": "init_dir" }, { "key": "\/apisix\/plugin_metadata\/clickhouse-logger", "value": "{\"log_format\":{\"body_bytes_sent\":\"$body_bytes_sent\",\"route_name\":\"$route_name\",\"time_local\":\"$time_local\",\"http_host\":\"$http_host\",\"time\":\"$time_iso8601\",\"upstream_uri\":\"$upstream_uri\",\"request_query\":\"$args\",\"request_body\":\"$request_body\",\"method\":\"$request_method\",\"upstream_addr\":\"$upstream_addr\",\"request_proto\":\"$server_protocol\",\"http_referrer\":\"$name\",\"upstream_host\":\"$upstream_host\",\"x_forward_for\":\"$proxy_add_x_forwarded_for\",\"unix_time\":\"$msec\",\"http_user_agent\":\"$http_user_agent\",\"upstream_scheme\":\"$upstream_scheme\",\"request\":\"$request\",\"request_time\":\"$request_time\",\"upstream_status\":\"$upstream_status\",\"vhost\":\"$host\",\"remote_addr\":\"$proxy_protocol_addr\",\"path\":\"$uri\",\"remote_user\":\"$remote_user\"}}" }, { "key": "\/apisix\/plugins", "value": "[{\"name\":\"api-breaker\"},{\"name\":\"authz-casbin\"},{\"name\":\"authz-casdoor\"},{\"name\":\"authz-keycloak\"},{\"name\":\"aws-lambda\"},{\"name\":\"azure-functions\"},{\"name\":\"basic-auth\"},{\"name\":\"batch-requests\"},{\"name\":\"clickhouse-logger\"},{\"name\":\"client-control\"},{\"name\":\"consumer-restriction\"},{\"name\":\"cors\"},{\"name\":\"csrf\"},{\"name\":\"datadog\"},{\"name\":\"dubbo-proxy\"},{\"name\":\"echo\"},{\"name\":\"error-log-logger\"},{\"name\":\"example-plugin\"},{\"name\":\"ext-plugin-post-req\"},{\"name\":\"ext-plugin-post-resp\"},{\"name\":\"ext-plugin-pre-req\"},{\"name\":\"fault-injection\"},{\"name\":\"file-logger\"},{\"name\":\"forward-auth\"},{\"name\":\"google-cloud-logging\"},{\"name\":\"grpc-transcode\"},{\"name\":\"grpc-web\"},{\"name\":\"gzip\"},{\"name\":\"hmac-auth\"},{\"name\":\"http-logger\"},{\"name\":\"ip-restriction\"},{\"name\":\"jwt-auth\"},{\"name\":\"kafka-logger\"},{\"name\":\"kafka-proxy\"},{\"name\":\"key-auth\"},{\"name\":\"ldap-auth\"},{\"name\":\"limit-conn\"},{\"name\":\"limit-count\"},{\"name\":\"limit-req\"},{\"name\":\"log-rotate\"},{\"name\":\"loggly\"},{\"name\":\"mocking\"},{\"name\":\"node-status\"},{\"name\":\"opa\"},{\"name\":\"openid-connect\"},{\"name\":\"opentelemetry\"},{\"name\":\"openwhisk\"},{\"name\":\"prometheus\"},{\"name\":\"proxy-control\"},{\"name\":\"proxy-mirror\"},{\"name\":\"proxy-rewrite\"},{\"name\":\"public-api\"},{\"name\":\"real-ip\"},{\"name\":\"redirect\"},{\"name\":\"referer-restriction\"},{\"name\":\"request-id\"},{\"name\":\"request-validation\"},{\"name\":\"response-rewrite\"},{\"name\":\"rocketmq-logger\"},{\"name\":\"server-info\"},{\"name\":\"serverless-post-function\"},{\"name\":\"serverless-pre-function\"},{\"name\":\"sls-logger\"},{\"name\":\"splunk-hec-logging\"},{\"name\":\"syslog\"},{\"name\":\"tcp-logger\"},{\"name\":\"traffic-split\"},{\"name\":\"ua-restriction\"},{\"name\":\"udp-logger\"},{\"name\":\"uri-blocker\"},{\"name\":\"wolf-rbac\"},{\"name\":\"zipkin\"},{\"name\":\"jwt-for-1ren\"},{\"stream\":true,\"name\":\"ip-restriction\"},{\"stream\":true,\"name\":\"limit-conn\"},{\"stream\":true,\"name\":\"mqtt-proxy\"},{\"stream\":true,\"name\":\"syslog\"}]" }, { "key": "\/apisix\/plugins\/", "value": "init_dir" }, { "key": "\/apisix\/proto\/", "value": "init_dir" }, { "key": "\/apisix\/routes\/", "value": "init_dir" }, { "key": "\/apisix\/routes\/414615654771983046", "value": "{\"id\":\"414615654771983046\",\"create_time\":1656659788,\"update_time\":1657688527,\"uri\":\"\/escort\/collect\/ws\",\"name\":\"dev_escort_collect_ws\",\"desc\":\"护驾测试环境采集模块长链接\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\"}},\"service_id\":\"414615505169547974\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"enable_websocket\":true,\"status\":1}" }, { "key": "\/apisix\/routes\/414622106198737606", "value": "{\"id\":\"414622106198737606\",\"create_time\":1656663633,\"update_time\":1657256578,\"uris\":[\"\/dashboard\/\",\"\/apisix\/\"],\"name\":\"dev_apisix_dashboard_h5\",\"desc\":\"dashboard\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"upstream\":{\"nodes\":{\"172.16.5.189:9001\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}},\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"apisix\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414628375827579590", "value": "{\"id\":\"414628375827579590\",\"create_time\":1656667370,\"update_time\":1657688577,\"uri\":\"\/escort\/collect\/\",\"name\":\"dev_escort_collect\",\"desc\":\"护驾测试环境采集模块接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\",\"regex_uri\":[\"\/escort\/collect\/\",\"\/\"]}},\"service_id\":\"414622816210518726\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414728833116144326", "value": "{\"id\":\"414728833116144326\",\"create_time\":1656727247,\"update_time\":1657784068,\"uri\":\"\/escortCommunity\/\",\"name\":\"dev_escort_community_app_interface\",\"desc\":\"护驾测试环境社区模块app接口\",\"priority\":1,\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"ssl_verify\":false,\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\",\"regex_uri\":[\"\/escortCommunity\/\",\"\/\"]}},\"service_id\":\"414728522402104006\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414729016054907590", "value": "{\"id\":\"414729016054907590\",\"create_time\":1656727356,\"update_time\":1657688651,\"uri\":\"\/escort\/web\/escortCommunity\/\",\"name\":\"dev_escort_community_web_interface\",\"desc\":\"护驾测试环境社区模块web接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\",\"regex_uri\":[\"\/escort\/web\/escortCommunity\/\",\"\/\"]}},\"service_id\":\"414728672440746694\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414729982036673222", "value": "{\"id\":\"414729982036673222\",\"create_time\":1656727932,\"update_time\":1657853036,\"uri\":\"\/escort\/service\/\",\"name\":\"dev_escort_service_interface\",\"desc\":\"护驾测试环境业务模块接口\",\"priority\":1,\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"regex_uri\":[\"\/escort\/service\/\",\"\/\"]}},\"service_id\":\"414729848473256646\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414730310064800454", "value": "{\"id\":\"414730310064800454\",\"create_time\":1656728128,\"update_time\":1657702182,\"uri\":\"\/escort\/info\/active\/\",\"name\":\"dev_escort_active_interface\",\"desc\":\"护驾测试环境活动模块接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\"}},\"service_id\":\"414730110919246534\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414731217343087302", "value": "{\"id\":\"414731217343087302\",\"create_time\":1656728668,\"update_time\":1657702234,\"uri\":\"\/escort\/info\/energy\/\",\"name\":\"dev_escort_energy_interface\",\"desc\":\"护驾测试环境补能模块接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\"}},\"service_id\":\"414731099516699334\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414731449472647878", "value": "{\"id\":\"414731449472647878\",\"create_time\":1656728807,\"update_time\":1657702014,\"uri\":\"\/escort\/crm\/\",\"name\":\"dev_escort_crm_interface\",\"desc\":\"护驾测试环境crm接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\"}},\"service_id\":\"414731018969285318\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414731734316221126", "value": "{\"id\":\"414731734316221126\",\"create_time\":1656728977,\"update_time\":1657870946,\"uri\":\"\/escort\/web\/\",\"name\":\"dev_escort_service_web_interface\",\"desc\":\"护驾测试环境业务模块web接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\",\"regex_uri\":[\"\/escort\/web\/\",\"\/\"]}},\"service_id\":\"414731624811332294\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414732255399772870", "value": "{\"id\":\"414732255399772870\",\"create_time\":1656729287,\"update_time\":1657689070,\"uri\":\"\/escort\/oss-sts\",\"name\":\"dev_escort_ossSts\",\"desc\":\"护驾测试环境OSS-STS\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"proxy-rewrite\":{\"host\":\"172.16.5.174\",\"uri\":\"\/\"}},\"service_id\":\"414732092056797894\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/414734439306756806", "value": "{\"id\":\"414734439306756806\",\"create_time\":1656730589,\"update_time\":1657869731,\"uri\":\"\/escort\/h5\/\",\"name\":\"dev_escort_home_h5\",\"desc\":\"护驾测试环境H5\",\"priority\":1,\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"service_id\":\"414734269001237190\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415016511887901382", "value": "{\"id\":\"415016511887901382\",\"create_time\":1656898717,\"update_time\":1657764587,\"uri\":\"\/\",\"name\":\"dev_apisix_dashboard_status\",\"desc\":\"dashboard静态文件\",\"priority\":2,\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"vars\":[[\"uri\",\"~~\",\".(js|css|flash|media|jpg|png|gif|ico|vbs|json|txt|svg)$\"]],\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.189\",\"port\":9001,\"weight\":1}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}},\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"apisix\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415053802706567878", "value": "{\"id\":\"415053802706567878\",\"create_time\":1656920944,\"update_time\":1657689152,\"uri\":\"\/escort\/static\/\",\"name\":\"dev_escort_service_static\",\"desc\":\"护驾测试环境H5静态资源文件\t\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"service_id\":\"415053596615246534\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415057043343278790", "value": "{\"id\":\"415057043343278790\",\"create_time\":1656922876,\"update_time\":1657689196,\"uri\":\"\/escort\/service\/escortCommunity\/\",\"name\":\"dev_escort_community_share\",\"desc\":\"护驾测试环境分享接口\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"plugins\":{\"clickhouse-logger\":{\"database\":\"default\",\"endpoint_addr\":\"http:\/\/172.16.5.189:8133\",\"logtable\":\"escort_logger\",\"password\":\"pd4lJf0Eha1OUPzWUJ\",\"user\":\"default\"},\"proxy-rewrite\":{\"host\":\"172.16.5.187\",\"regex_uri\":[\"\/escort\/service\/escortCommunity\/\",\"\/\"]}},\"service_id\":\"414728522402104006\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415057404321858246", "value": "{\"id\":\"415057404321858246\",\"create_time\":1656923091,\"update_time\":1657764003,\"uri\":\"\/escort\/admin\/\",\"name\":\"dev_escort_web_dashboard\",\"desc\":\"护驾测试环境WEB页面\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"service_id\":\"415054510755414726\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415065285234524870", "value": "{\"id\":\"415065285234524870\",\"create_time\":1656927788,\"update_time\":1657791576,\"uri\":\"\/escort\/energyH5\/\",\"name\":\"dev_escort_energy_h5\",\"desc\":\"护驾测试环境补能页面\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"service_id\":\"414734269001237190\",\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"escort-test\"},\"status\":1}" }, { "key": "\/apisix\/routes\/415173139194970822", "value": "{\"id\":\"415173139194970822\",\"create_time\":1656992074,\"update_time\":1656992074,\"uri\":\"\/loki\/api\/v1\/push\",\"name\":\"dev_loki_promtail_interface\",\"desc\":\"loki接收日志\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"pay.1rendata.com\",\"upstream\":{\"nodes\":{\"172.16.5.189:3100\":1},\"retries\":2,\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320},\"retry_timeout\":2},\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"apisix\"},\"status\":1}" }, { "key": "\/apisix\/routes\/416332727281779398", "value": "{\"id\":\"416332727281779398\",\"create_time\":1657683243,\"update_time\":1657850105,\"uri\":\"\/grafana\/\",\"name\":\"grafana\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"mydomain.com\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.189\",\"port\":3000,\"weight\":1}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}},\"status\":0}" }, { "key": "\/apisix\/routes\/416612663888446149", "value": "{\"id\":\"416612663888446149\",\"create_time\":1657850098,\"update_time\":1657850122,\"uri\":\"\/grafana\/\",\"name\":\"dev_loki_grafana_web\",\"desc\":\"grafana日志监控可视化\",\"methods\":[\"GET\",\"POST\",\"PUT\",\"DELETE\",\"PATCH\",\"HEAD\",\"OPTIONS\",\"CONNECT\",\"TRACE\"],\"host\":\"pay.1rendata.com\",\"upstream\":{\"nodes\":{\"172.16.5.189:3000\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}},\"labels\":{\"API_VERSION\":\"v1.0.0\",\"project\":\"apisix\"},\"status\":1}" }, { "key": "\/apisix\/services\/", "value": "init_dir" }, { "key": "\/apisix\/services\/414615505169547974", "value": "{\"id\":\"414615505169547974\",\"create_time\":1656659699,\"update_time\":1657531143,\"name\":\"护驾测试环境采集模块长链接\",\"desc\":\"护驾测试环境采集模块长链接\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":9001,\"weight\":10000}],\"timeout\":{\"connect\":120,\"send\":120,\"read\":120},\"type\":\"roundrobin\",\"scheme\":\"https\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":100000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414622816210518726", "value": "{\"id\":\"414622816210518726\",\"create_time\":1656664056,\"update_time\":1656667945,\"name\":\"护驾测试环境采集模块接口服务\",\"desc\":\"护驾测试环境采集模块接口服务\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":20003,\"weight\":1}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414629460826915526", "value": "{\"id\":\"414629460826915526\",\"create_time\":1656668017,\"update_time\":1656668017,\"name\":\"护驾测试环境xxl-job\",\"desc\":\"护驾测试环境xxl-job\",\"upstream\":{\"nodes\":{\"172.16.5.187:8090\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414728522402104006", "value": "{\"id\":\"414728522402104006\",\"create_time\":1656727062,\"update_time\":1657531154,\"name\":\"护驾测试环境社区APP接口\",\"desc\":\"护驾测试环境社区APP接口\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":20006,\"weight\":1000000}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414728672440746694", "value": "{\"id\":\"414728672440746694\",\"create_time\":1656727151,\"update_time\":1656727151,\"name\":\"护驾测试环境社区模块web接口\",\"desc\":\"护驾测试环境社区模块web接口\",\"upstream\":{\"nodes\":{\"172.16.5.187:20005\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414729848473256646", "value": "{\"id\":\"414729848473256646\",\"create_time\":1656727852,\"update_time\":1656728371,\"name\":\"护驾测试环境业务模块接口\",\"desc\":\"护驾测试环境业务模块接口\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":19001,\"weight\":1}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414730110919246534", "value": "{\"id\":\"414730110919246534\",\"create_time\":1656728009,\"update_time\":1656728378,\"name\":\"护驾测试环境活动模块接口\",\"desc\":\"护驾测试环境活动模块接口\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":19003,\"weight\":1}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414731018969285318", "value": "{\"id\":\"414731018969285318\",\"create_time\":1656728550,\"update_time\":1657531177,\"name\":\"护驾测试环境crm接口\",\"desc\":\"护驾测试环境crm接口\",\"upstream\":{\"nodes\":[{\"host\":\"172.16.5.187\",\"port\":19002,\"weight\":1000000}],\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414731099516699334", "value": "{\"id\":\"414731099516699334\",\"create_time\":1656728598,\"update_time\":1656728598,\"name\":\"护驾测试环境补能模块接口\",\"desc\":\"护驾测试环境补能模块接口\",\"upstream\":{\"nodes\":{\"172.16.5.187:19004\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414731624811332294", "value": "{\"id\":\"414731624811332294\",\"create_time\":1656728911,\"update_time\":1656728911,\"name\":\"护驾测试环境业务模块web接口\",\"desc\":\"护驾测试环境业务模块web接口\",\"upstream\":{\"nodes\":{\"172.16.5.187:19005\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414732092056797894", "value": "{\"id\":\"414732092056797894\",\"create_time\":1656729190,\"update_time\":1656729190,\"name\":\"护驾测试环境OSS-STS\",\"desc\":\"护驾测试环境OSS-STS\",\"upstream\":{\"nodes\":{\"172.16.5.174:7080\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/414734269001237190", "value": "{\"id\":\"414734269001237190\",\"create_time\":1656730487,\"update_time\":1657869578,\"name\":\"护驾测试环境APPH5\",\"desc\":\"护驾测试环境APPH5\",\"upstream\":{\"nodes\":{\"172.16.5.185:80\":1},\"retries\":3,\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320},\"retry_timeout\":10}}" }, { "key": "\/apisix\/services\/415053596615246534", "value": "{\"id\":\"415053596615246534\",\"create_time\":1656920821,\"update_time\":1656920821,\"name\":\"护驾测试环境H5静态资源文件\",\"desc\":\"护驾测试环境H5静态资源文件\",\"upstream\":{\"nodes\":{\"172.16.5.185:80\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/services\/415054510755414726", "value": "{\"id\":\"415054510755414726\",\"create_time\":1656921366,\"update_time\":1656921366,\"name\":\"护驾测试环境WEB页面\",\"desc\":\"护驾测试环境WEB页面\",\"upstream\":{\"nodes\":{\"172.16.5.185:80\":1},\"timeout\":{\"connect\":6,\"send\":6,\"read\":6},\"type\":\"roundrobin\",\"scheme\":\"http\",\"pass_host\":\"pass\",\"keepalive_pool\":{\"idle_timeout\":60,\"requests\":1000,\"size\":320}}}" }, { "key": "\/apisix\/ssl\/", "value": "init_dir" }, { "key": "\/apisix\/stream_routes\/", "value": "init_dir" }, { "key": "\/apisix\/upstreams\/", "value": "init_dir" } ]

1renwuruiming commented 2 years ago

@tzssangglass 我从etcd导出了json数据，并脱敏了相关的信息，请看一下这样的路由提供方式是否有用，感谢

tzssangglass commented 2 years ago

@tzssangglass 我从etcd导出了json数据，并脱敏了相关的信息，请看一下这样的路由提供方式是否有用，感谢

ok, I will check this issue and #7136 together.

tzssangglass commented 2 years ago

My recurrence process is as follows:

apisix version 2.13.1

apisix-base version: nginx version: openresty/1.21.4.1

  1. upstream

use openresty mock upstream, config like:

master_process on;

worker_processes 2;

error_log logs/error.log warn;
pid logs/nginx.pid;

worker_rlimit_nofile 20480;

events {
    accept_mutex off;
    worker_connections 10620;
}

worker_shutdown_timeout 3;

http {
    server {
        listen 1980;
        location /escort/collect/ {
            content_by_lua_block {
                ngx.say("/escort/collect/")
            }
        }

        location /escort/collect/ws {
            content_by_lua_block {
                ngx.say("/escort/collect/ws")
            }
        }

        location /escort/web/ {
            content_by_lua_block {
                ngx.say("/escort/web/")
            }
        }
    }
}

and test

# hit /escort/collect/
$ curl http://127.0.0.1:1980/escort/collect/
/escort/collect/

$ curl http://127.0.0.1:1980/escort/collect/1
/escort/collect/

# hit /escort/collect/ws
$ curl http://127.0.0.1:1980/escort/collect/ws
/escort/collect/ws

$ curl http://127.0.0.1:1980/escort/collect/ws/
/escort/collect/ws

# hit /escort/web/
$ curl http://127.0.0.1:1980/escort/web/
/escort/web/

$ curl http://127.0.0.1:1980/escort/web/1
/escort/web/

As you can see, the route matching here follows the style of Nginx.

  1. route in APISIX
curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
{
  "uri":"/escort/collect/",
  "plugins":{
      "proxy-rewrite":{
          "regex_uri":[
              "/escort/collect/",
              "/"
          ]
      }
  },
  "upstream":{
      "type":"roundrobin",
      "nodes":{
          "127.0.0.1:1980":1
      }
  }
}'
  1. test proxy by APISIX
curl http://127.0.0.1:9080/escort/collect/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>openresty/1.21.4.1</center>
</body>
</html>

here 404 status was returned by upstream, why? because the request URI was changed by proxy-rewrite from /escort/collect/ to /, the upstream doesn't have this location.

apisix version 2.13.2

apisix-base version: nginx version: openresty/1.21.4.1

Repeat the above process, and get consistent results.

possible errors

APISIX route matching differs from Nginx's location

Your routing configuration is as follows:

According to the above test cases, if the request is curl http://127.0.0.1:9080/escort/collect/, would match the route1, and if the request is curl http://127.0.0.1:9080/escort/collect/1, would get:

 curl http://127.0.0.1:9080/escort/collect/1
{"error_msg":"404 Route Not Found"}

this means that APISIX cannot match the request, it does not match /escort/collect/ locations like Nginx does.

ref: https://github.com/apache/apisix/blob/master/docs/en/latest/terminology/route.md ref: https://github.com/apache/apisix/blob/master/docs/en/latest/terminology/router.md

proxy-rewrite

You need to verify whether it is the 404 Route Not Found returned by APISIX or the 404 returned by the upstream.

You should know exactly which route the request will be caught by and how the request URI will be modified by proxy-rewrite.

reproduce your case

You can describe what you think the routing chaos looks like, based on my reproduction case modification above.

cc @1renwuruiming @Fatpa

tzssangglass commented 2 years ago

And for reproducibility, you can test by providing just one route describing the difference between 2.13.1 and 2.13.2 in the same scenario (same route configuration, same upstream), which will allow us to pinpoint the problem faster.

tzssangglass commented 2 years ago

Or you think that if two routes routeA and routeB exist at the same time, and the same request is matched by routeA at 2.13.1 and by routeB at 2.13.2, then you can set up two routes.

1renwuruiming commented 2 years ago

如果是upstream的路由不存在，那么为什么会时好时坏呢？在没有修改过apisix的route的情况下，如果上游路由地址不对，那应该始终是404，不是么

fatpa commented 2 years ago

First, we should know what mainly changed between 2.13.1 and 2.13.2?

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

I still believe that the simple test case can't find out the problem of this case.

fatpa commented 2 years ago

During the upgrade processing, we don't change any routes config, all the errors should not lead by the route config. Maybe some performance problem exists?

1renwuruiming commented 2 years ago

我将会重试2.14.1版本时的路由跳转问题，2.13.2版本我未曾进行过验证

tzssangglass commented 2 years ago

First, we should know what mainly changed between 2.13.1 and 2.13.2?

here is the commit history: https://github.com/apache/apisix/commits/release/2.13

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

in APISIX, /* would match any request URI, So I don't think 404 will be generated by APISIX, we should focus on what happens between the request being proxied by APISIX and going upstream, which is unknown to me.

I still believe that the simple test case can't find out the problem of this case.

So can you give steps that can be reproduced and accurately describe the anomaly as I have given? I need to reproduce it in my environment in order to locate it.

tzssangglass commented 2 years ago

我将会重试2.14.1版本时的路由跳转问题，2.13.2版本我未曾进行过验证

I also tried it in 2.14.1 and the conclusion is the same.

tzssangglass commented 2 years ago

如果是upstream的路由不存在，那么为什么会时好时坏呢？在没有修改过apisix的route的情况下，如果上游路由地址不对，那应该始终是404，不是么

So can you give details of the steps that can reproduce this phenomenon?

fatpa commented 2 years ago

First, we should know what mainly changed between 2.13.1 and 2.13.2?

here is the commit history: https://github.com/apache/apisix/commits/release/2.13

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

in APISIX, /* would match any request URI, So I don't think 404 will be generated by APISIX, we should focus on what happens between the request being proxied by APISIX and going upstream, which is unknown to me.

I still believe that the simple test case can't find out the problem of this case.

So can you give steps that can be reproduced and accurately describe the anomaly as I have given? I need to reproduce it in my environment in order to locate it.

你得测试验证高并发场景下的 case，比如对一个 route 做 ab benchmark。

1renwuruiming commented 2 years ago

从2.14.1降级到2.13.1时，发现http_access_phase()会提示错误，两个版本之间这里应该有不同吧 image

tzssangglass commented 2 years ago

从2.14.1降级到2.13.1时，发现http_access_phase()会提示错误，两个版本之间这里应该有不同吧

Do not post pictures, use text.

1renwuruiming commented 2 years ago

First, we should know what mainly changed between 2.13.1 and 2.13.2?

here is the commit history: https://github.com/apache/apisix/commits/release/2.13

And in my test case, my route setting is simple, only /*, and it doesn't happen in a single curl test, but in high concurrency scenes. And in the mean time, some of requests is work fine.

in APISIX, /* would match any request URI, So I don't think 404 will be generated by APISIX, we should focus on what happens between the request being proxied by APISIX and going upstream, which is unknown to me.

I still believe that the simple test case can't find out the problem of this case.

So can you give steps that can be reproduced and accurately describe the anomaly as I have given? I need to reproduce it in my environment in order to locate it.

你得测试验证高并发场景下的 case，比如对一个 route 做 ab benchmark。

我在请求并发不高的情况下可以轻而易举的复现出这个问题，当然，apisix的版本是2.14.1

1renwuruiming commented 2 years ago

从2.14.1降级到2.13.1时，发现http_access_phase()会提示错误，两个版本之间这里应该有不同吧

Do not post pictures, use text.

2022/07/15 01:42:56 [error] 45#45: *118 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:362: attempt to index upvalue 'local_conf' (a nil value) stack traceback:coroutine 0: /usr/local/apisix/apisix/init.lua: in function 'http_access_phase' access_by_lua(nginx.conf:302):2: in main chunk,

tzssangglass commented 2 years ago

我在请求并发不高的情况下可以轻而易举的复现出这个问题，当然，apisix的版本是2.14.1

Then give a detailed step by step that I can reproduce, based on 2.14.1

1renwuruiming commented 2 years ago

我在请求并发不高的情况下可以轻而易举的复现出这个问题，当然，apisix的版本是2.14.1

Then give a detailed step by step that I can reproduce, based on 2.14.1

我将重新安装2.14.1版本的apisix，并重试这个问题，如果我找到了系统的复现过程，我将会提供出来

tzssangglass commented 2 years ago

2022/07/15 01:42:56 [error] 45#45: *118 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:362: attempt to index upvalue 'local_conf' (a nil value) stack traceback:coroutine 0: /usr/local/apisix/apisix/init.lua: in function 'http_access_phase' access_by_lua(nginx.conf:302):2: in main chunk,

Please submit another issue for this and give the steps to reproduce it, especially in detail, until then you can look at: https://github.com/apache/apisix/issues/3830

It may have more to do with the way you switch versions.

1renwuruiming commented 2 years ago

2022/07/15 01:42:56 [error] 45#45: *118 lua entry thread aborted: runtime error: /usr/local/apisix/apisix/init.lua:362: attempt to index upvalue 'local_conf' (a nil value) stack traceback:coroutine 0: /usr/local/apisix/apisix/init.lua: in function 'http_access_phase' access_by_lua(nginx.conf:302):2: in main chunk,

Please submit another issue for this and give the steps to reproduce it, especially in detail, until then you can look at: #3830

It may have more to do with the way you switch versions.

好的，那么大概清楚了，谢谢

github-actions[bot] commented 1 year ago

This issue has been marked as stale due to 350 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.

github-actions[bot] commented 1 year ago

This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.