www_fdw crash with postgres 10

[ioanadanes]

Hi,

We recently switched to postgres 10 and www_fdw causes a postgres crash with:

2018-08-22 10:51:07.714 EDT,,,16723,,5b7c43a7.4153,119,,2018-08-21 12:53:59 EDT,,0,LOG,00000,"server process (PID 26661) was terminated by signal 11: Segmentation fault","Failed process was running: select * from www_fdw.test_balance where retailerid = 1025",,,,,,,,"" 2018-08-22 10:51:07.714 EDT,,,16723,,5b7c43a7.4153,120,,2018-08-21 12:53:59 EDT,,0,LOG,00000,"terminating any other active server processes",,,,,,,,,"" 2018-08-22 10:51:07.721 EDT,,,16723,,5b7c43a7.4153,121,,2018-08-21 12:53:59 EDT,,0,LOG,00000,"all server processes terminated; reinitializing",,,,,,,,,"" 2018-08-22 10:51:07.751 EDT,,,26730,,5b7d785b.686a,1,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"database system was interrupted; last known up at 2018-08-22 00:03:46 EDT",,,,,,,,,"" 2018-08-22 10:51:07.911 EDT,,,26731,"10.20.2.39:61696",5b7d785b.686b,1,"",2018-08-22 10:51:07 EDT,,0,LOG,00000,"connection received: host=10.20.2.39 port=61696",,,,,,,,,"" 2018-08-22 10:51:07.916 EDT,"postgres","postgres",26731,"10.20.2.39:61696",5b7d785b.686b,2,"",2018-08-22 10:51:07 EDT,,0,FATAL,57P03,"the database system is in recovery mode",,,,,,,,,"" 2018-08-22 10:51:09.411 EDT,,,26733,"10.20.2.39:61699",5b7d785d.686d,1,"",2018-08-22 10:51:09 EDT,,0,LOG,00000,"connection received: host=10.20.2.39 port=61699",,,,,,,,,"" 2018-08-22 10:51:09.422 EDT,"postgres","ioana_test",26733,"10.20.2.39:61699",5b7d785d.686d,2,"",2018-08-22 10:51:09 EDT,,0,FATAL,57P03,"the database system is in recovery mode",,,,,,,,,"" 2018-08-22 10:51:10.103 EDT,,,26734,"10.20.2.39:61701",5b7d785e.686e,1,"",2018-08-22 10:51:10 EDT,,0,LOG,00000,"connection received: host=10.20.2.39 port=61701",,,,,,,,,"" 2018-08-22 10:51:10.104 EDT,"postgres","ioana_test",26734,"10.20.2.39:61701",5b7d785e.686e,2,"",2018-08-22 10:51:10 EDT,,0,FATAL,57P03,"the database system is in recovery mode",,,,,,,,,"" 2018-08-22 10:51:10.220 EDT,,,26730,,5b7d785b.686a,2,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"database system was not properly shut down; automatic recovery in progress",,,,,,,,,"" 2018-08-22 10:51:10.228 EDT,,,26730,,5b7d785b.686a,3,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"redo starts at B/8BC9D448",,,,,,,,,"" 2018-08-22 10:51:10.232 EDT,,,26730,,5b7d785b.686a,4,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"invalid record length at B/8BD07000: wanted 24, got 0",,,,,,,,,"" 2018-08-22 10:51:10.232 EDT,,,26730,,5b7d785b.686a,5,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"redo done at B/8BD06FA8",,,,,,,,,"" 2018-08-22 10:51:10.232 EDT,,,26730,,5b7d785b.686a,6,,2018-08-22 10:51:07 EDT,,0,LOG,00000,"last completed transaction was at log time 2018-08-22 10:50:54.762065-04",,,,,,,,,""

Environment: CentOS Linux release 7.4.1708 (Core) Postgres 10.4 www_fdw 0.1.9

I use response_deserialize_callback:

CREATE SERVER test_balance_server FOREIGN DATA WRAPPER www_fdw OPTIONS (uri '', response_type 'other', response_deserialize_callback 'www_fdw.test_balance_response_deserialize_callback', ssl_cert '/data01/pki/server-cert.pem', ssl_key '/data01/pki/server-key.pem', cainfo '/data01/pki/ca-cert.pem', proxy '');

and the crash happens only when I use a where condition. I noticed it crashes in the function serialize_request_parameters at: List *quals = list_copy(node->ss.ps.qual);

and in the list_copy function at the line: newlist = new_list(oldlist->type);

I use version 0.1.8 with postgres 9.4 and I have no issued with it.

Hany help is greatly appreciated.

Thanks, Ioana

[leonmvd]

Same problem here on MacOs

[kaoecoito]

I did some tests and got it right for version 10 There have been changes in postgresql for some of the functions used so a simple way to get the code working is by changing the line in function serialize_request_parameters.

List *quals = list_copy(node->ss.ps.qual);

to

List *quals = list_copy(node->ss.ps.plan->qual);

and the line

char *param = www_param((Node *) state->expr, to char *param = www_param((Node *) state,

[kaoecoito]

Complementing! Missed line change

node->ss.ps.qual = list_delete(node->ss.ps.qual, (void *) state);

to node->ss.ps.plan->qual = list_delete(node->ss.ps.plan->qual, (void *) state);