Crash in validate mode on regex matches in routes since 1.23

[sankethb27]

Title: Crash in validate mode on regex matches in routes since 1.23

Description: Hi,

We’ve identified a crash that occurs in validate mode (and only validate mode) when a route includes a regex match against a header. There appears to be no issue when mode is set to ‘serve’. Please find attached a static config and a Dockerfile (with the same config baked in) for reproduction.

We’ve tested this case in various versions and it appears to have been introduced in 1.23 and remains present in all current 1.24 releases.

Any other kind of string matcher works, and the deprecated google_re2 matcher also works (with the expected deprecation warning). The only issue appears to be the new ‘regex’ option introduced in 1.23.0

Dockerfile:

FROM envoyproxy/envoy:v1.24.1

RUN echo 'static_resources:\n\
  listeners:\n\
  - address:\n\
      socket_address:\n\
        address: "127.0.0.1"\n\
        port_value: 80\n\
    filter_chains:\n\
      filters:\n\
      - name: envoy.http_connection_manager\n\
        typed_config:\n\
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager\n\
          stat_prefix: example\n\
          route_config:\n\
            name: demo\n\
            virtual_hosts:\n\
            - name: default\n\
              domains:\n\
              - "*"\n\
              routes:\n\
              - match:\n\
                  prefix: "/"\n\
                  headers:\n\
                    - name: "X-Whatever"\n\
                      string_match:\n\
                        safe_regex:\n\
                            regex: ".*"\n\
                direct_response:\n\
                  status: 200' > /etc/envoy/envoy.yaml

CMD ["envoy", "-c", "/etc/envoy/envoy.yaml", "--mode", "validate"]

Sample envoy template:

static_resources:
  listeners:
  - address:
      socket_address:
        address: "127.0.0.1"
        port_value: 80
    filter_chains:
      filters:
      - name: envoy.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: example
          route_config:
            name: demo
            virtual_hosts:
            - name: default
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                  headers:
                    - name: "X-Whatever"
                      string_match:
                        safe_regex:
                          regex: ".*"
                direct_response:
                  status: 200

Sample output from the issue:

[2022-12-22 16:33:56.805][1][info][main] [source/server/server.cc:808] runtime: {}
[2022-12-22 16:33:56.805][1][info][config] [source/server/configuration_impl.cc:131] loading tracing configuration
[2022-12-22 16:33:56.805][1][info][config] [source/server/configuration_impl.cc:91] loading 0 static secret(s)
[2022-12-22 16:33:56.805][1][info][config] [source/server/configuration_impl.cc:97] loading 0 cluster(s)
[2022-12-22 16:33:56.806][1][info][config] [source/server/configuration_impl.cc:101] loading 1 listener(s)
[2022-12-22 16:33:56.807][1][critical][assert] [./source/common/singleton/threadsafe_singleton.h:56] assert failure: loader_ != nullptr. Details: InjectableSingleton used prior to initialization
[2022-12-22 16:33:56.807][1][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x6500000001
[2022-12-22 16:33:56.807][1][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2022-12-22 16:33:56.807][1][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: 69958e4fe32da561376d8b1d367b5e6942dfba24/1.24.1/Clean/RELEASE/BoringSSL
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:96] #0: __kernel_rt_sigreturn [0xffffa3cf0790]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:96] #1: abort [0xffffa3a65aac]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #2: [0xaaaad48524e0]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #3: [0xaaaad4851bd4]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #4: [0xaaaad62e407c]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #5: [0xaaaad4974ad8]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #6: [0xaaaad6060e4c]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #7: [0xaaaad6068c00]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #8: [0xaaaad607f42c]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #9: [0xaaaad606a740]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #10: [0xaaaad6069d64]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #11: [0xaaaad606b8e8]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #12: [0xaaaad606d350]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #13: [0xaaaad5fc1e5c]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #14: [0xaaaad5fc8e78]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #15: [0xaaaad5fbcd14]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #16: [0xaaaad5fc03c0]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #17: [0xaaaad5fc77b4]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #18: [0xaaaad5fbe6e4]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #19: [0xaaaad5fbc9c4]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #20: [0xaaaad5f37a50]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #21: [0xaaaad5f36504]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #22: [0xaaaad5ed1438]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #23: [0xaaaad5edb350]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #24: [0xaaaad5edafb0]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #25: [0xaaaad5ee3d88]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #26: [0xaaaad5ec3c54]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #27: [0xaaaad5ed6da0]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #28: [0xaaaad5ed6028]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #29: [0xaaaad5f2b7f0]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #30: [0xaaaad5acdbe8]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #31: [0xaaaad5accd58]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #32: [0xaaaad5acc24c]
[2022-12-22 16:33:56.808][1][critical][backtrace] [./source/server/backtrace.h:98] #33: [0xaaaad46f02e4]
[2022-12-22 16:33:56.809][1][critical][backtrace] [./source/server/backtrace.h:98] #34: [0xaaaad46f09c8]
[2022-12-22 16:33:56.809][1][critical][backtrace] [./source/server/backtrace.h:98] #35: [0xaaaad46ed13c]
[2022-12-22 16:33:56.809][1][critical][backtrace] [./source/server/backtrace.h:96] #36: __libc_start_main [0xffffa3a65e10]

[doujiang24]

Seems https://github.com/envoyproxy/envoy/pull/24037 fixed this bug

[sankethb27]

@doujiang24, Was this fix released as part for the latest envoy version? I'm currently using v1.24.1 and still seeing the same error.

[doujiang24]

@sankethb27 I'm afraid not. It's merged in this commit, which is only in the main branch.

[sankethb27]

@doujiang24, Is there any ETA on when this is going to be available?

[doujiang24]

@sankethb27 Sorry, I'm not familiar with the release schedule. Maybe maintainers could give you an answer.

[phlax]

/backport

[phlax]

hi - regarding releases - the schedule is here https://github.com/envoyproxy/envoy/blob/main/RELEASES.md

patch releases happen as required and/or time available

[phlax]

backport PRs raised:

• 24821

• 24822