search

Segfault-Inc / Multicorn

Data Access Library
https://multicorn.org/
PostgreSQL License
700 stars 145 forks source link

Regression failure with python3.8: write_filesystem #245

Closed df7cb closed 4 years ago

df7cb commented 4 years ago

With PG 9.6: (other versions likely affected as well)

https://pgdgbuild.dus.dg-i.net/job/postgresql-multicorn-binaries/architecture=amd64,distribution=sid/80/consoleFull

07:34:01 **** regression.diffs ****
07:34:01 --- /tmp/autopkgtest.xjiAiH/tree/test-3.8/expected/write_filesystem.out    2019-10-18 08:37:04.000000000 +0000
07:34:01 +++ /tmp/autopkgtest.xjiAiH/tree/results/write_filesystem.out  2020-03-03 06:34:00.446606169 +0000
07:34:01 @@ -143,45 +143,39 @@
07:34:01  -- Insertion with incoherent filename/properties (should fail)
07:34:01  INSERT INTO testmulticorn (color, size, name, ext, data, filename) VALUES ('blue', 'big', 'triangle', 'txt', 'Im a big blue triangle',
07:34:01          'blue/small/triangle.txt');
07:34:01 -psql:test-common/multicorn_testfilesystem.include:28: ERROR:  The columns inferred from the filename do not match the supplied columns.
07:34:01 -HINT:  Remove either the filename column or the properties column from your  statement, or ensure they match
07:34:01  -- Insertion with missing keys (should fail)
07:34:01  INSERT INTO testmulticorn (color, size, name) VALUES ('blue', 'small', 'triangle');
07:34:01 -psql:test-common/multicorn_testfilesystem.include:31: ERROR:  The following columns are necessary: {'ext'}
07:34:01 -HINT:  You can also insert an item by providing only the filename and content columns
07:34:01  -- Insertion with missing keys and filename (should fail)
07:34:01  INSERT INTO testmulticorn (color, size, name, filename) VALUES ('blue', 'small', 'triangle', 'blue/small/triangle.txt');
07:34:01 -psql:test-common/multicorn_testfilesystem.include:34: ERROR:  The following columns are necessary: {'ext'}
07:34:01 -HINT:  You can also insert an item by providing only the filename and content columns
07:34:01  -- Insertion which would overwrite a file.
07:34:01  -- Normal insertion
07:34:01  INSERT INTO testmulticorn (color, size, name, ext, data) VALUES ('yellow', 'big', 'square', 'text', 'Im a duplicate big square');
07:34:01 -psql:test-common/multicorn_testfilesystem.include:38: ERROR:  Duplicate key value violates filesystem integrity.
07:34:01 -DETAIL:  Key (color, ext, name, size)=(yellow, text, square, big) already exists
07:34:01  -- Should still have 11 lines by now.
07:34:01  SELECT * from testmulticorn ORDER BY filename;
07:34:01 - color  | size  |   name   | ext  |        filename         |           data           
07:34:01 ---------+-------+----------+------+-------------------------+--------------------------
07:34:01 - blue   | big   | round    | ini  | blue/big/round.ini      | Im a big blue round     +
07:34:01 -        |       |          |      |                         | 
07:34:01 - blue   | big   | square   | txt  | blue/big/square.txt     | Im a big blue square    +
07:34:01 -        |       |          |      |                         | 
07:34:01 - blue   | big   | triangle | txt  | blue/big/triangle.txt   | Im a big blue triangle
07:34:01 - blue   | small | round    | ini  | blue/small/round.ini    | Im a small blue round   +
07:34:01 -        |       |          |      |                         | 
07:34:01 - blue   | small | square   | txt  | blue/small/square.txt   | Im a small blue square  +
07:34:01 -        |       |          |      |                         | 
07:34:01 - red    | big   | round    | ini  | red/big/round.ini       | Im a big red round      +
07:34:01 -        |       |          |      |                         | 
07:34:01 - red    | big   | square   | txt  | red/big/square.txt      | Im a big red square     +
07:34:01 -        |       |          |      |                         | 
07:34:01 - red    | small | round    | ini  | red/small/round.ini     | Im a small red round    +
07:34:01 -        |       |          |      |                         | 
07:34:01 - red    | small | square   | txt  | red/small/square.txt    | Im a small red square   +
07:34:01 -        |       |          |      |                         | 
07:34:01 - yellow | big   | square   | text | yellow/big/square.text  | Im a big yellow square
07:34:01 - yellow | small | square   | txt  | yellow/small/square.txt | Im a small yellow square
07:34:01 -(11 rows)
07:34:01 + color  | size  |   name   | ext  |         filename         |           data            
07:34:01 +--------+-------+----------+------+--------------------------+---------------------------
07:34:01 + blue   | big   | round    | ini  | blue/big/round.ini       | Im a big blue round      +
07:34:01 +        |       |          |      |                          | 
07:34:01 + blue   | big   | square   | txt  | blue/big/square.txt      | Im a big blue square     +
07:34:01 +        |       |          |      |                          | 
07:34:01 + blue   | big   | triangle | txt  | blue/big/triangle.txt    | Im a big blue triangle
07:34:01 + blue   | small | round    | ini  | blue/small/round.ini     | Im a small blue round    +
07:34:01 +        |       |          |      |                          | 
07:34:01 + blue   | small | square   | txt  | blue/small/square.txt    | Im a small blue square   +
07:34:01 +        |       |          |      |                          | 
07:34:01 + blue   | small | triangle | None | blue/small/triangle.None | 
07:34:01 + blue   | small | triangle | txt  | blue/small/triangle.txt  | 
07:34:01 + red    | big   | round    | ini  | red/big/round.ini        | Im a big red round       +
07:34:01 +        |       |          |      |                          | 
07:34:01 + red    | big   | square   | txt  | red/big/square.txt       | Im a big red square      +
07:34:01 +        |       |          |      |                          | 
07:34:01 + red    | small | round    | ini  | red/small/round.ini      | Im a small red round     +
07:34:01 +        |       |          |      |                          | 
07:34:01 + red    | small | square   | txt  | red/small/square.txt     | Im a small red square    +
07:34:01 +        |       |          |      |                          | 
07:34:01 + yellow | big   | square   | text | yellow/big/square.text   | Im a duplicate big square
07:34:01 + yellow | small | square   | txt  | yellow/small/square.txt  | Im a small yellow square
07:34:01 +(13 rows)
07:34:01  
07:34:01  -- Test insertion in transaction
07:34:01  BEGIN;
07:34:01 @@ -230,30 +224,27 @@
07:34:01  
07:34:01  -- Update should not work if it would override an existing file.
07:34:01  UPDATE testmulticorn set filename = 'blue/big/triangle.txt' where filename = 'blue/big/rectangle.txt';
07:34:01 -psql:test-common/multicorn_testfilesystem.include:65: ERROR:  Duplicate key value violates filesystem integrity.
07:34:01 -DETAIL:  Key (color, ext, name, size)=(blue, txt, triangle, big) already exists
07:34:01  -- Update should not work when setting filename column to NULL
07:34:01  UPDATE testmulticorn set filename = NULL where filename = 'blue/big/rectangle.txt';
07:34:01 -psql:test-common/multicorn_testfilesystem.include:68: ERROR:  The filename, or all pattern columns are needed.
07:34:01  -- Update should not work when setting a property column to NULL
07:34:01  WITH t as (
07:34:01    UPDATE testmulticorn set color = NULL where filename = 'blue/big/rectangle.txt' RETURNING color
07:34:01  )
07:34:01  SELECT * from t ORDER BY color;
07:34:01 -psql:test-common/multicorn_testfilesystem.include:73: ERROR:  Null value in columns (color) are not allowed
07:34:01 -DETAIL:  Failing row contains (NULL, big, rectangle, txt)
07:34:01 + color 
07:34:01 +-------
07:34:01 +(0 rows)
07:34:01 +
07:34:01  -- Content column update.
07:34:01  UPDATE testmulticorn set data = 'Im an updated rectangle' where filename = 'blue/big/rectangle.txt' RETURNING data;
07:34:01 -          data           
07:34:01 --------------------------
07:34:01 - Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + data 
07:34:01 +------
07:34:01 +(0 rows)
07:34:01  
07:34:01  SELECT * from testmulticorn where filename = 'blue/big/rectangle.txt';
07:34:01 - color | size |   name    | ext |        filename        |          data           
07:34:01 --------+------+-----------+-----+------------------------+-------------------------
07:34:01 - blue  | big  | rectangle | txt | blue/big/rectangle.txt | Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + color | size | name | ext | filename | data 
07:34:01 +-------+------+------+-----+----------+------
07:34:01 +(0 rows)
07:34:01  
07:34:01  -- Update in transactions
07:34:01  BEGIN;
07:34:01 @@ -269,7 +260,7 @@
07:34:01      SELECT count(1) from testmulticorn where name = 'square';
07:34:01   count 
07:34:01  -------
07:34:01 -     6
07:34:01 +     5
07:34:01  (1 row)
07:34:01  
07:34:01  ROLLBACK;
07:34:01 @@ -284,7 +275,7 @@
07:34:01  SELECT count(1) from testmulticorn where name = 'rectangle';
07:34:01   count 
07:34:01  -------
07:34:01 -     6
07:34:01 +     5
07:34:01  (1 row)
07:34:01  
07:34:01  BEGIN;
07:34:01 @@ -293,13 +284,14 @@
07:34:01      SELECT count(1) from testmulticorn where data ilike '% UPDATED!';
07:34:01   count 
07:34:01  -------
07:34:01 -    11
07:34:01 +    12
07:34:01  (1 row)
07:34:01  
07:34:01      SELECT data from testmulticorn where data ilike '% UPDATED!' order by filename limit 1;
07:34:01 -               data               
07:34:01 -----------------------------------
07:34:01 - Im an updated rectangle UPDATED!
07:34:01 +        data         
07:34:01 +---------------------
07:34:01 + Im a big blue round+
07:34:01 +  UPDATED!
07:34:01  (1 row)
07:34:01  
07:34:01  ROLLBACK;
07:34:01 @@ -314,62 +306,65 @@
07:34:01      UPDATE testmulticorn set data = data || ' UPDATED!';
07:34:01      UPDATE testmulticorn set data = data || ' TWICE!';
07:34:01      SELECT data from testmulticorn order by filename; 
07:34:01 -                   data                   
07:34:01 -------------------------------------------
07:34:01 - Im an updated rectangle UPDATED! TWICE!
07:34:01 - Im a big blue round                     +
07:34:01 +                   data                    
07:34:01 +-------------------------------------------
07:34:01 + Im a big blue round                      +
07:34:01 +  UPDATED! TWICE!
07:34:01 + Im a big blue square                     +
07:34:01 +  UPDATED! TWICE!
07:34:01 + Im a small blue square                   +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a big blue triangle UPDATED! TWICE!
07:34:01 - Im a small blue square                  +
07:34:01 + Im a small blue round                    +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a small blue round                   +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a big red square                     +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a big red round                      +
07:34:01 + Im a big red square                      +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a small red square                   +
07:34:01 + Im a big red round                       +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a small red round                    +
07:34:01 + Im a small red square                    +
07:34:01    UPDATED! TWICE!
07:34:01 - Im a big yellow square UPDATED! TWICE!
07:34:01 + Im a small red round                     +
07:34:01 +  UPDATED! TWICE!
07:34:01 + Im a duplicate big square UPDATED! TWICE!
07:34:01   Im a small yellow square UPDATED! TWICE!
07:34:01 -(11 rows)
07:34:01 +(12 rows)
07:34:01  
07:34:01  ROLLBACK;
07:34:01  -- No 'UPDATED! or 'TWICE!'
07:34:01  SELECT data from testmulticorn order by filename;
07:34:01 -           data           
07:34:01 ---------------------------
07:34:01 - Im an updated rectangle
07:34:01 - Im a big blue round     +
07:34:01 +           data            
07:34:01 +---------------------------
07:34:01 + Im a big blue round      +
07:34:01 + 
07:34:01 + Im a big blue square     +
07:34:01 + 
07:34:01 + Im a small blue square   +
07:34:01 + 
07:34:01 + Im a small blue round    +
07:34:01   
07:34:01 - Im a big blue triangle
07:34:01 - Im a small blue square  +
07:34:01   
07:34:01 - Im a small blue round   +
07:34:01   
07:34:01 - Im a big red square     +
07:34:01 + Im a big red square      +
07:34:01   
07:34:01 - Im a big red round      +
07:34:01 + Im a big red round       +
07:34:01   
07:34:01 - Im a small red square   +
07:34:01 + Im a small red square    +
07:34:01   
07:34:01 - Im a small red round    +
07:34:01 + Im a small red round     +
07:34:01   
07:34:01 - Im a big yellow square
07:34:01 + Im a duplicate big square
07:34:01   Im a small yellow square
07:34:01 -(11 rows)
07:34:01 +(12 rows)
07:34:01  
07:34:01  -- Test successive update to the same files.
07:34:01  BEGIN;
07:34:01      UPDATE testmulticorn set color = 'cyan' where filename = 'blue/big/rectangle.txt';
07:34:01      -- There should be one line with cyan color, 0 with the old filename
07:34:01      SELECT filename, data from testmulticorn where color = 'cyan' order by filename;
07:34:01 -        filename        |          data           
07:34:01 -------------------------+-------------------------
07:34:01 - cyan/big/rectangle.txt | Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + filename | data 
07:34:01 +----------+------
07:34:01 +(0 rows)
07:34:01  
07:34:01      SELECT filename, data from testmulticorn where filename = 'blue/big/rectangle.txt' order by filename;
07:34:01   filename | data 
07:34:01 @@ -380,10 +375,9 @@
07:34:01      -- filename
07:34:01      UPDATE testmulticorn set color = 'magenta' where color = 'cyan';
07:34:01      SELECT filename, data from testmulticorn where color = 'magenta' order by filename;  
07:34:01 -         filename          |          data           
07:34:01 ----------------------------+-------------------------
07:34:01 - magenta/big/rectangle.txt | Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + filename | data 
07:34:01 +----------+------
07:34:01 +(0 rows)
07:34:01  
07:34:01      SELECT filename, data from testmulticorn where color = 'cyan' order by filename;
07:34:01   filename | data 
07:34:01 @@ -398,10 +392,9 @@
07:34:01      UPDATE testmulticorn set color = 'blue' where color = 'magenta';
07:34:01      -- There should be one line with the old filename, and zero with the rest
07:34:01      SELECT filename, data from testmulticorn where filename = 'blue/big/rectangle.txt' order by filename;
07:34:01 -        filename        |          data           
07:34:01 -------------------------+-------------------------
07:34:01 - blue/big/rectangle.txt | Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + filename | data 
07:34:01 +----------+------
07:34:01 +(0 rows)
07:34:01  
07:34:01      SELECT filename, data from testmulticorn where color = 'magenta' order by filename;
07:34:01   filename | data 
07:34:01 @@ -416,10 +409,9 @@
07:34:01  COMMIT;
07:34:01  -- Result should be the same than pre-commit
07:34:01  SELECT filename, data from testmulticorn where filename = 'blue/big/rectangle.txt' order by filename;
07:34:01 -        filename        |          data           
07:34:01 -------------------------+-------------------------
07:34:01 - blue/big/rectangle.txt | Im an updated rectangle
07:34:01 -(1 row)
07:34:01 + filename | data 
07:34:01 +----------+------
07:34:01 +(0 rows)
07:34:01  
07:34:01  SELECT filename, data from testmulticorn where color = 'magenta' order by filename;
07:34:01   filename | data 
07:34:01
df7cb commented 4 years ago

Fwiw this is the same as #195 which I opened 2.5 years ago. ffs.

rdunklau commented 4 years ago

This should be fixed on master.

df7cb commented 4 years ago

It's indeed fixed, thanks!. Closing now.