help request: when using `hmac-auth` and `proxy-rewrite` plugin simultaneously, why does the rewrite phase of `proxy-rewrite` gets executed twice? Is this a Bug?

Current Behavior

In apisix release/2.13 and release/3.2, when using hmac-auth and proxy-rewrite plugins simultaneously on a route, the rewrite phase of proxy-rewrite is observed to be executed twice.

Expected Behavior

No response

Error Logs

2024/01/29 13:04:36 [warn] 26073#26073: 2 [lua] plugin.lua:252: load_stream(): new plugins: {"mqtt-proxy":true,"ip-restriction":true,"limit-conn":true,"syslog":true}, context: init_worker_by_lua 2024/01/29 13:04:36 [warn] 26071#26071: 60 [lua] config_etcd.lua:168: main etcd watcher started, revision=1374, context: ngx.timer 2024/01/29 13:04:36 [warn] 26074#26074: 80 [lua] config_etcd.lua:168: main etcd watcher started, revision=1374, context: ngx.timer 2024/01/29 13:04:36 [warn] 26073#26073: 93 [lua] config_etcd.lua:168: main etcd watcher started, revision=1374, context: ngx.timer 2024/01/29 13:05:29 [warn] 26073#26073: 3478 [lua] proxy-rewrite.lua:261: phasefunc(): 开始》》》 proxy rewrite, client: 127.0.0.1, server: , request: "GET /api/v1/books?id=2 HTTP/1.1", host: "127.0.0.1:27151" 2024/01/29 13:05:29 [warn] 26073#26073: 3478 [lua] proxy-rewrite.lua:351: phasefunc(): 结束 << << << proxy rewrite, client: 127.0.0.1, server: , request: "GET /api/v1/books?id=2 HTTP/1.1", host: "127.0.0.1:27151" 2024/01/29 13:05:29 [warn] 26073#26073: 3478 [lua] proxy-rewrite.lua:261: phasefunc(): 开始》》》 proxy rewrite, client: 127.0.0.1, server: , request: "GET /api/v1/books?id=2 HTTP/1.1", host: "127.0.0.1:27151" 2024/01/29 13:05:29 [warn] 26073#26073: *3478 [lua] proxy-rewrite.lua:351: phasefunc(): 结束 << << << proxy rewrite, client: 127.0.0.1, server: , request: "GET /api/v1/books?id=2 HTTP/1.1", host: "127.0.0.1:27151"

Steps to Reproduce

1、run with apisix source code

Add debu log in proxy_rewrite plugin

function _M.rewrite(conf, ctx)
    core.log.warn("开始 》》》 proxy rewrite")
    for _, name in ipairs(upstream_names) do
        if conf[name] then
            ctx.var[upstream_vars[name]] = conf[name]
        end
    end
    ......
    if conf.method then
        ngx.req.set_method(switch_map[conf.method])
    end
    core.log.warn("结束 << << << proxy rewrite")
end

config.yaml

deployment:
  role: traditional
  role_traditional:
    config_provider: etcd
  admin:
    admin_key:
      - name: admin
        key: ebca3b7b5508638c15ba670c8c7963c7  # 使用实例ID的md5值
        role: admin
    allow_admin:
    admin_listen:
      ip: 0.0.0.0
      port: 27152
  etcd:
    host:
      - http://127.0.0.1:27157

apisix:
  node_listen:
    - port: 27151 # 网关的监听端口
      # enable_http2: true
  ssl:
    listen:
      - port: 27154 # 网关ssl的监听端口
        # enable_http2: true
  enable_control: true
  control:
    ip: "0.0.0.0"
    port: 27153  # 网关控制api的端口
  proxy_cache:         #缓存插件配置
    cache_ttl: 60s     # 若上游未指定过期时间，则为默认缓存时间
    zones:
      - name: memory_cache
        memory_size: 50m

plugin_attr:
  prometheus:
    metric_prefix: mse_gw_
    export_addr:
      ip: 0.0.0.0
      port: 9091
    username: prometheus
    password: ENC(U2FsdGVkX1+yy1B8J1Zn35mHCDj7bX8GcTChRQeEWjo=)
  opentelemetry:
    resource:
      service.name: MSEGW
      project.code: MSEGW
      deployment: dev
      instance.id : "802635138818114560"
    collector:
      address: 127.0.0.1:27159
      request_timeout: 3

nginx_config:
  user: root

plugins:                          # plugin list (sorted by priority)
  - real-ip                        # priority: 23000
  - client-control                 # priority: 22000
  - proxy-control                  # priority: 21990
  - zipkin                         # priority: 12011
  #- skywalking                    # priority: 12010
  - opentelemetry                 # priority: 12009
  - ext-plugin-pre-req             # priority: 12000
  - request-id                     # priority: 11010
  - fault-injection                # priority: 11000
  - mocking                        # priority: 10900
  - serverless-pre-function        # priority: 10000
  #- batch-requests                # priority: 4010
  - cors                           # priority: 4000
  - ip-restriction                 # priority: 3000
  - ua-restriction                 # priority: 2999
  - referer-restriction            # priority: 2990
  - csrf                           # priority: 2980
  - uri-blocker                    # priority: 2900
  - request-validation             # priority: 2800
  - openid-connect                 # priority: 2599
  - authz-casbin                   # priority: 2560
  - wolf-rbac                      # priority: 2555
  - ldap-auth                      # priority: 2540
  - hmac-auth                      # priority: 2530
  - basic-auth                     # priority: 2520
  - jwt-auth                       # priority: 2510
  - key-auth                       # priority: 2500
  - consumer-restriction           # priority: 2400
  - forward-auth                   # priority: 2002
  - opa                            # priority: 2001
  - authz-keycloak                 # priority: 2000
  #- error-log-logger              # priority: 1091
  - proxy-mirror                   # priority: 1010
  - proxy-cache                    # priority: 1009
  - proxy-rewrite                  # priority: 1008
  - api-breaker                    # priority: 1005
  - limit-conn                     # priority: 1003
  - limit-count                    # priority: 1002
  - limit-req                      # priority: 1001
  #- node-status                   # priority: 1000
  - gzip                           # priority: 995
  - server-info                    # priority: 990
  - traffic-split                  # priority: 966
  - redirect                       # priority: 900
  - response-rewrite               # priority: 899
  #- dubbo-proxy                   # priority: 507
  - grpc-transcode                 # priority: 506
  - grpc-web                       # priority: 505
  - public-api                     # priority: 501
  - prometheus                     # priority: 500
  - datadog                        # priority: 495
  - echo                           # priority: 412
  - loggly                         # priority: 411
  - http-logger                    # priority: 410
  - splunk-hec-logging             # priority: 409
  - skywalking-logger              # priority: 408
  - google-cloud-logging           # priority: 407
  - sls-logger                     # priority: 406
  - tcp-logger                     # priority: 405
  - kafka-logger                   # priority: 403
  - rocketmq-logger                # priority: 402
  - syslog                         # priority: 401
  - udp-logger                     # priority: 400
  - file-logger                    # priority: 399
  - clickhouse-logger              # priority: 398
  #- log-rotate                    # priority: 100
  # <- recommend to use priority (0, 100) for your custom plugins
  - example-plugin                 # priority: 0
  - aws-lambda                     # priority: -1899
  - azure-functions                # priority: -1900
  - openwhisk                      # priority: -1901
  - serverless-post-function       # priority: -2000
  - ext-plugin-post-req            # priority: -3000

2、http-client.env

{
  "apisix-test": {
    "host": "127.0.0.1",
    "listenPort": "27151",
    "adminPort": "27152",
    "controlPort": "27153",
    "X-API-KEY": "ebca3b7b5508638c15ba670c8c7963c7",
    "upstreamUrl": "apisix/admin/upstreams",
    "routeUrl": "apisix/admin/routes",
    "resourceId": "1979"
  }
}

3、create upstream

A book review demo

### create upstream
PUT http://{{host}}:{{adminPort}}/{{upstreamUrl}}/{{resourceId}}
X-API-KEY: {{X-API-KEY}}
Content-Type: application/json

{
  "type":"roundrobin",
  "nodes":{
    "127.0.0.1:39087": 1
  }
}

4、create consumer

### create consumer
PUT {{host}}:{{adminPort}}/apisix/admin/consumers/
X-API-KEY: {{X-API-KEY}}
Content-Type: application/json

{
    "username": "7471377788",
    "plugins": {
        "hmac-auth": {
            "access_key": "7471377788",
            "encode_uri_params": true,
            "validate_request_body": false,
            "max_req_body": 524288,
            "algorithm": "hmac-sha256",
            "clock_skew": 0,
            "keep_headers": false,
            "secret_key": "bd9facb9e981449a82ff2331865abd1c"
        }
    }
}

5、create route wih hmac-auth and proxy-rewrite plugin

### create route
PUT {{host}}:{{adminPort}}/apisix/admin/routes/{{resourceId}}
X-API-KEY: {{X-API-KEY}}
Content-Type: application/json

{
    "uri": "/api/v1/books",
    "upstream_id": "{{resourceId}}",
    "vars": [["http_test1", "==", "1"]],
    "priority": 2,
    "id": "{{resourceId}}",
    "name": "test",
    "status": 1,
    "plugins": {
        "proxy-rewrite": {
            "uri": "\/api\/v1\/books",
            "disable": false,
            "path_match_type": "EQUAL"
        },
        "hmac-auth": {
            "disable": false
        }
    }
}

6、request this route

### request
GET http://{{host}}:{{listenPort}}/api/v1/books?id=2
X-HMAC-ACCESS-KEY: 7471377788
X-HMAC-SIGNATURE: EN6AkEta1fJ0E2NUaLaaS6A6XKcaJC7xco5jRjARzDU=
X-HMAC-ALGORITHM: hmac-sha256
test1: 1

Environment

APISIX version (run apisix version): release/2.13 or release/3.2
Operating system (run uname -a): Linux 0000000g-ZDXrgOHSNP 5.4.0-149-generic 18.04.1-Ubuntu SMP Fri Apr 21 16:42:44 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
OpenResty / Nginx version (run openresty -V or nginx -V): nginx version: openresty/1.25.3.1 built with OpenSSL 1.1.1w 11 Sep 2023 TLS SNI support enabled
etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): v3.5.1
APISIX Dashboard version, if relevant: no
Plugin runner version, for issues related to plugin runners: hmac-auth（version=0.1）、proxy-rewrite（version=0.1）
LuaRocks version, for installation issues (run luarocks --version): 3.8.0

apache / apisix