citusdata / citus

Distributed PostgreSQL as an extension
https://www.citusdata.com
GNU Affero General Public License v3.0
10.1k stars 650 forks source link

Citus planner assumes that WHERE clause entries have a column reference for all JOINs #751

Closed onderkalaci closed 7 years ago

onderkalaci commented 7 years ago

I hit the issue while playing with #741.

The issue is that SelectClauseTableIdList() function assumes that all WHERE clause entries should refer to a column. However, that is not always the case.

Here are the steps to reprdouce the issue:

-- create and distribute table
CREATE TABLE main_table (user_id int, username text, timeout bigint, occuruence_time date);
SELECT master_create_distributed_table('main_table', 'user_id', 'hash');
SELECT master_create_worker_shards('main_table', 16, 2);

CREATE TABLE reference_table (user_id int, username text, timeout bigint, occuruence_time date);
SELECT master_create_distributed_table('reference_table', 'user_id', 'hash');
SELECT master_create_worker_shards('reference_table', 16, 2);

CREATE OR REPLACE FUNCTION simpleTestFunction(regclass)
    RETURNS text AS
$$
DECLARE
    strresult text;
BEGIN
    RETURN md5($1::text);
END;
$$
LANGUAGE 'plpgsql' IMMUTABLE;

CREATE OR REPLACE FUNCTION simpleTestFunction2()
    RETURNS text AS
$$
DECLARE
    strresult text;
BEGIN
    RETURN md5('text');
END;
$$
LANGUAGE 'plpgsql' IMMUTABLE;

SELECT 
   * 
FROM 
   main_table, reference_table 
WHERE 
    simpleTestFunction('main_table') = '%test%';

server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

SELECT 
   * 
FROM
    main_table, reference_table
WHERE 
    main_table.user_id = reference_table.user_id AND 
    simpleTestFunction('main_table') = '%test%';

server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

SELECT 
   * 
FROM 
    main_table, reference_table 
WHERE
    simpleTestFunction2() = '%test%';

server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

SELECT 
  * 
FROM
   main_table, reference_table
WHERE
    main_table.user_id = reference_table.user_id AND 
   simpleTestFunction2() = '%test%';

server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> 

-- Also, try with RANDOM() function
SELECT 
   * 
FROM 
   main_table, reference_table 
WHERE 
   main_table.user_id = reference_table.user_id AND random() > 0.5;

server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
onderkalaci commented 7 years ago

Also, @mtuncer noted that we should check what happens when WHERE clause entry includes more than one column references.

ozgune commented 7 years ago

@onderkalaci -- This issue looks related to #804. @mtuncer / @marcocitus -- would you agree?

The problem is that the logical planner/optimizer plan for relational algebra and random() > 0.5 doesn't have any relations. Ideally, we'd track non-relational filters in a separate select clause list and not use them during logical planning (only refer to them during physical planning). For the examples you've provided however, we may need to take it one step further and evaluate them at a later stage.

@mtuncer / @onderkalaci -- We earlier said that FALSE clauses are the only non-relational filters. It looks like there are more. What's our current behavior for these clauses? If we're crashing, should we at least error out?

marcocitus commented 7 years ago

This was fixed by #804