pgpool / pgpool2

This is the official mirror of git://git.postgresql.org/git/pgpool2.git. Note that this is just a *mirror* - we don't work with pull requests on github. Please subscribe to pgpool-hackers mailing list from our website and submit your patch to this mailing list.
https://www.pgpool.net
Other
339 stars 89 forks source link

Segfault in pgpool when executing a specific SQL query #54

Closed hiroin closed 5 months ago

hiroin commented 5 months ago

[Overview]
Attempting to execute the SQL query in the attached file segfault_sql.txt on pgpool causes a segmentation fault.

[Environment]

$ cat /etc/almalinux-release
AlmaLinux release 8.9 (Midnight Oncilla)

$ uname -r
4.18.0-477.13.1.el8_8.x86_64

$ pgpool -v
pgpool-II version 4.5.2 (hotooriboshi)

$ sudo cat /etc/pgpool-II/pgpool.conf | grep backend_clustering_mode
backend_clustering_mode = 'streaming_replication'

# SELECT version();
                                                 version
---------------------------------------------------------------------------------------------------------
 PostgreSQL 15.7 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20), 64-bit
(1 row)

[Details]
When executing the SQL query from segfault_sql.txt via psql on pgpool, the following occurs:

$ psql -p 9999
Null display is "¤".
psql (15.7)
Type "help" for help.

user=# select *
user-# from
user-# (with
(omitted)
user(# ) as tmpr;
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.

The server's messages log shows the following:

Jun  1 19:45:03 sakura kernel: pgpool[270143]: segfault at 0 ip 00005576cfa379fa sp 00007ffc3d84aea0 error 4 in pgpool[5576cf9ab000+245000]
Jun  1 19:45:03 sakura kernel: Code: 8b 7c 24 08 41 8b 46 40 4c 89 ff 8d 50 ff 49 8b 86 88 00 00 00 48 8d 70 01 e8 32 f6 ff ff 48 89 c1 48 89 44 24 08 49 8b 47 70 <48> 8b 00 48 85 c0 0f 84 c0 01 00 00 49 8b 57 78 31 f6 48 89 cf ff

A core dump was generated, and a backtrace using gdb shows the following:

$ sudo gdb /usr/bin/pgpool /var/tmp/core/core.pgpool.270143
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-20.el8
...
Core was generated by `pgpool: ap2 fanza  SELECT                       '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00005576cfa379fa in psql_yylex (yylval_param=yylval_param@entry=0x0, yyscanner=<optimized out>)
    at utils/psqlscan.l:711
711                                             if (cur_state->callbacks->get_variable)
...
(gdb) bt
#0  0x00005576cfa379fa in psql_yylex (yylval_param=yylval_param@entry=0x0, yyscanner=<optimized out>)
    at utils/psqlscan.l:711
#1  0x00005576cfa383c7 in psql_scan (state=state@entry=0x5576d0bf6268, query_buf=query_buf@entry=0x7ffc3d84afa0,
    prompt=prompt@entry=0x7ffc3d84af9c) at utils/psqlscan.l:1149
#2  0x00005576cf9fd1be in multi_statement_query (
    queries=0x5576d0c34438 "select *\nfrom\n(with\n  okazuconf as (select * from (values\n  (  4, 4), /*とても使えた*/\n  (  3, 3), /*かなり使えた*/\n  (  2, 2), /*だいぶ使えた*/\n  (  1, 1), /*それなりに使", <incomplete sequence \343\201>...) at protocol/pool_proto_modules.c:4839
#3  SimpleQuery (frontend=frontend@entry=0x5576d0bef848, backend=backend@entry=0x7f875181b228, len=10710,
    contents=contents@entry=0x5576d0c34438 "select *\nfrom\n(with\n  okazuconf as (select * from (values\n  (  4, 4), /*とても使えた*/\n  (  3, 3), /*かなり使えた*/\n  (  2, 2), /*だいぶ使えた*/\n  (  1, 1), /*それなりに使", <incomplete sequence \343\201>...) at protocol/pool_proto_modules.c:300
#4  0x00005576cfa01320 in ProcessFrontendResponse (frontend=frontend@entry=0x5576d0bef848,
    backend=backend@entry=0x7f875181b228) at protocol/pool_proto_modules.c:2819
#5  0x00005576cf9f1f88 in read_packets_and_process (frontend=<optimized out>, backend=<optimized out>,
    reset_request=<optimized out>, state=<optimized out>, num_fields=<optimized out>, cont=<optimized out>)
    at protocol/pool_process_query.c:5117
#6  0x00005576cf9f2913 in pool_process_query (frontend=<optimized out>, backend=<optimized out>,
    reset_request=<optimized out>) at protocol/pool_process_query.c:299
#7  0x00005576cf9eaaf7 in do_child (fds=fds@entry=0x5576d0c2a470) at protocol/child.c:467
#8  0x00005576cf9bc607 in fork_a_child (fds=0x5576d0c2a470, id=34) at main/pgpool_main.c:863
#9  0x00005576cf9c542c in service_child_processes () at main/pgpool_main.c:5096
#10 PgpoolMain (discard_status=<optimized out>, clear_memcache_oidmaps=<optimized out>) at main/pgpool_main.c:703
#11 0x00005576cf9ba57c in main (argc=<optimized out>, argv=0x7ffc3d85ab98) at main/main.c:365

The segmentation fault occurs during the SQL parsing stage in pgpool, and it can be reproduced regardless of the schema. Executing the query in segfault_sql.txt on any schema will trigger the segmentation fault, even if the tables specified in the query do not exist in the schema.

It was also confirmed that the same SQL query can be executed directly on PostgreSQL without any issues.

[Steps to Reproduce]

  1. Connect to pgpool using psql on any database.
  2. Execute the SQL query in the attached segfault_sql.txt file.

Attachments

Please let me know if you need any further information.

tatsuo-ishii commented 5 months ago

Thank you for the report. I was able to reproduce the segfault using your SQL sample. I think what happens here is:

  1. Since the query is large (over 10k), pgpool tries to find if the query is multi statement or not by using psqlscan.
  2. For some reason I don't know psqlscan wants to analyze if psql variable is included or not.
  3. Since I forgot to disable the analysis, psqlscan calls non-existent function for that, and it ends up with segfault.

I think the fix is to disable the analysis by providing NULL pointer to psqlscan initialize function. Note that psqlscan was imported from psql and originally the analysis was necessary of course, but it is not for pgpool. fix_sqlscan.txt

Attached is the patch to fix the issue. If you like, please try it.

hiroin commented 5 months ago

I have applied the patch you provided and confirmed that no segmentation fault occurs. Thank you. Since this is not urgent, please include it in one of the upcoming releases at your convenience. I will now close this issue.

tatsuo-ishii commented 5 months ago

Thank you for testing! The fix will appear in next minor releases.