bug: `upstream_status` is empty after executing the `serverless-pre-function` plugin

[vortegatorres]

Current Behavior

After executing the serverless-pre-function plugin with a Lua script that modifies the response, the upstream_status value is empty. This leads to issues such as an incorrect http_status attribute in the span when the OpenTelemetry plugin is executed, as it relies on the upstream_status value.

Expected Behavior

Ensure upstream_status retains a correct value after executing a Lua script that calls the core.response.exit() function.

Error Logs

{"ts": "2024-07-23T08:07:17+00:00", "service": "apisix", "resp_body_size": "162", "host": "-------", "address": "-------", "request_length": "446", "method": "GET", "uri": "-----", "status": "401", "user_agent": "------", "resp_time": "0.002", "upstream_addr": "", "upstream_status": "", "traceparent": "00-dc0718f09c93b3c1c4e4e42f5de40e44-42eaf107d787a636-01","trace_id": "dc0718f09c93b3c1c4e4e42f5de40e44","span_id": "42eaf107d787a636"}

Steps to Reproduce

1. Start APISIX.
2. Configure logs to display upstream_status by including it in the nginx_config.http.access_log_format.
3. Add a serverless-pre-function plugin that modifies the response status code:

   curl http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '
   {
   "uri": "/your-endpoint",
   "plugins": {
       "serverless-pre-function": {
           "phase": "rewrite",
           "functions": ["return function() local core = require(\"apisix.core\"); core.response.exit(401); end"]
       }
   }
   }'

4. Check the logs and verify that upstream_status is empty.

Environment

• APISIX version (run apisix version): 3.7.0
• Operating system (run uname -a): x86_64 GNU/Linux
• OpenResty / Nginx version (run openresty -V or nginx -V): nginx version: openresty/1.21.4.2
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info): etcd-8.7.7

[zhoujiexiong]

@vortegatorres

upstream_status is set at apisix.http_balancer_phase, but `serverless-pre-function @ rewrite phase returning 401(>=200, for more to see below) will interrupt that flow.

If you want to modify the response from upstream, try serverless-xxx @ [header | body]_filter phase.

core.response.exit(401) -> ngx.exit(401)

restydoc -s ngx.exit

When "status >= 200" (i.e., "ngx.HTTP_OK" and above), it will interrupt
       the execution of the current request and return status code to Nginx.
...

[vortegatorres]

@vortegatorres

upstream_status is set at apisix.http_balancer_phase, but `serverless-pre-function @ rewrite phase returning 401(>=200, for more to see below) will interrupt that flow.

If you want to modify the response from upstream, try serverless-xxx @ [header | body]_filter phase.

core.response.exit(401) -> ngx.exit(401)

restydoc -s ngx.exit

When "status >= 200" (i.e., "ngx.HTTP_OK" and above), it will interrupt
       the execution of the current request and return status code to Nginx.
...

If I understand correctly, the [header|body]_filter phases modify the upstream response. In my case, I use the serverless-pre-function plugin to be called from the forward-auth plugin, and there is no upstream body to modify, and your suggestion didn't work. Is there a workaround for setting the upstream_status in this scenario?

[zhoujiexiong]

@vortegatorres upstream_status is set at apisix.http_balancer_phase, but `serverless-pre-function @ rewrite phase returning 401(>=200, for more to see below) will interrupt that flow. If you want to modify the response from upstream, try serverless-xxx @ [header | body]_filter phase.

core.response.exit(401) -> ngx.exit(401)

restydoc -s ngx.exit

When "status >= 200" (i.e., "ngx.HTTP_OK" and above), it will interrupt
       the execution of the current request and return status code to Nginx.
...

If I understand correctly, the [header|body]_filter phases modify the upstream response. In my case, I use the serverless-pre-function plugin to be called from the forward-auth plugin, and there is no upstream body to modify, and your suggestion didn't work. Is there a workaround for setting the upstream_status in this scenario?

Could you show the route/plugins config more similar with your scenario?

[vortegatorres]

@vortegatorres upstream_status is set at apisix.http_balancer_phase, but `serverless-pre-function @ rewrite phase returning 401(>=200, for more to see below) will interrupt that flow. If you want to modify the response from upstream, try serverless-xxx @ [header | body]_filter phase.

core.response.exit(401) -> ngx.exit(401)

restydoc -s ngx.exit

When "status >= 200" (i.e., "ngx.HTTP_OK" and above), it will interrupt
       the execution of the current request and return status code to Nginx.
...

If I understand correctly, the [header|body]_filter phases modify the upstream response. In my case, I use the serverless-pre-function plugin to be called from the forward-auth plugin, and there is no upstream body to modify, and your suggestion didn't work. Is there a workaround for setting the upstream_status in this scenario?

Could you show the route/plugins config more similar with your scenario?

I have a route that is authenticated using the forward-auth plugin pointing to another route using serverless-pre-function plugin:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: service
  namespace: service
spec:
  http:
    - name: service
      match:
        paths:
          - "/service"
      backends:
        - serviceName: service
          servicePort: 10001
      plugins:
        - name: auth
          enable: true
          config:
            uri: "http://127.0.0.1:9080/auth"
    - name: schemas
      match:
        paths:
          - "/service"

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: auth
spec:
  http:
    - name: auth
      match:
        paths:
          - "/auth"
      plugins:
        - name: serverless-pre-function
          enable: true
          config:
            phase: rewrite
            functions:
              - return function(conf, ctx)
                  local core = require("apisix.core");          

                  core.response.exit(401);

[vortegatorres]

Is there any update on this issue?