verdict-project / verdict

Interactive-Speed Analytics: 200x Faster, 200x Fewer Cluster Resources, Approximate Query Processing
http://verdictdb.org
Apache License 2.0
248 stars 66 forks source link

The table 'lineitem_x' is full #318

Closed solangepaz closed 5 years ago

solangepaz commented 5 years ago

Hi, When i use 50GB of TPC-H i have this error on create a lineitem scramble.

>>> verdict.sql('create scramble tpch1g.lineitem_x from tpch1g.lineitem') 10:05:33.454 [Thread-2] INFO o.v.c.ScramblingCoordinator - Starts to create a new UNIFORM scramble tpch1g.lineitem_x from tpch1g.lineitem 10:05:33.455 [Thread-2] INFO o.v.c.ScramblingCoordinator - Method: UNIFORM 10:05:33.458 [Thread-2] INFO o.v.c.ScramblingCoordinator - Relative size: 1.000000 (or equivalently, 100.0000 %) org.verdictdb.exception.VerdictDBDbmsException: Issued the following query: create tabletpch1g.lineitem_xpartition by list columns (verdictdbblock) (partition p0 values in (0), partition p1 values in (1), partition p2 values in (2), partition p3 values in (3), partition p4 values in (4), partition p5 values in (5), partition p6 values in (6), partition p7 values in (7), partition p8 values in (8), partition p9 values in (9), partition p10 values in (10), partition p11 values in (11), partition p12 values in (12), partition p13 values in (13), partition p14 values in (14), partition p15 values in (15), partition p16 values in (16), partition p17 values in (17), partition p18 values in (18), partition p19 values in (19), partition p20 values in (20), partition p21 values in (21), partition p22 values in (22), partition p23 values in (23), partition p24 values in (24), partition p25 values in (25), partition p26 values in (26), partition p27 values in (27), partition p28 values in (28), partition p29 values in (29), partition p30 values in (30), partition p31 values in (31), partition p32 values in (32), partition p33 values in (33), partition p34 values in (34), partition p35 values in (35), partition p36 values in (36), partition p37 values in (37), partition p38 values in (38), partition p39 values in (39), partition p40 values in (40), partition p41 values in (41), partition p42 values in (42), partition p43 values in (43), partition p44 values in (44), partition p45 values in (45), partition p46 values in (46), partition p47 values in (47), partition p48 values in (48), partition p49 values in (49), partition p50 values in (50), partition p51 values in (51), partition p52 values in (52), partition p53 values in (53), partition p54 values in (54), partition p55 values in (55), partition p56 values in (56), partition p57 values in (57), partition p58 values in (58), partition p59 values in (59), partition p60 values in (60), partition p61 values in (61), partition p62 values in (62), partition p63 values in (63), partition p64 values in (64), partition p65 values in (65), partition p66 values in (66), partition p67 values in (67), partition p68 values in (68), partition p69 values in (69), partition p70 values in (70), partition p71 values in (71), partition p72 values in (72), partition p73 values in (73), partition p74 values in (74), partition p75 values in (75), partition p76 values in (76), partition p77 values in (77), partition p78 values in (78), partition p79 values in (79), partition p80 values in (80), partition p81 values in (81), partition p82 values in (82), partition p83 values in (83), partition p84 values in (84), partition p85 values in (85), partition p86 values in (86), partition p87 values in (87), partition p88 values in (88), partition p89 values in (89), partition p90 values in (90), partition p91 values in (91), partition p92 values in (92), partition p93 values in (93), partition p94 values in (94), partition p95 values in (95), partition p96 values in (96), partition p97 values in (97), partition p98 values in (98), partition p99 values in (99)) select t.l_orderkey, t.l_partkey, t.l_suppkey, t.l_linenumber, t.l_quantity, t.l_extendedprice, t.l_discount, t.l_tax, t.l_returnflag, t.l_linestatus, t.l_shipdate, t.l_commitdate, t.l_receiptdate, t.l_shipinstruct, t.l_shipmode, t.l_comment, t.l_dummy, 0 asverdictdbtier, cast(floor(rand() * 100) as unsigned) asverdictdbblockfromtpch1g.lineitemas t The table 'lineitem_x' is full at org.verdictdb.connection.JdbcConnection.executeSingle(JdbcConnection.java:222) at org.verdictdb.connection.JdbcConnection.execute(JdbcConnection.java:182) at org.verdictdb.connection.CachedDbmsConnection.execute(CachedDbmsConnection.java:49) at org.verdictdb.core.execplan.ExecutableNodeRunner.execute(ExecutableNodeRunner.java:428) at org.verdictdb.core.execplan.ExecutableNodeRunner.executeAndBroadcast(ExecutableNodeRunner.java:411) at org.verdictdb.core.execplan.ExecutableNodeRunner.run(ExecutableNodeRunner.java:339) at java.lang.Thread.run(Thread.java:748) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/ubuntu/miniconda3/lib/python3.7/site-packages/pyverdict/verdictcontext.py", line 93, in sql return self.sql_raw_result(query).to_df() File "/home/ubuntu/miniconda3/lib/python3.7/site-packages/pyverdict/verdictcontext.py", line 101, in sql_raw_result java_resultset = self._context.sql(query) File "/home/ubuntu/miniconda3/lib/python3.7/site-packages/py4j/java_gateway.py", line 1286, in __call__ answer, self.gateway_client, self.target_id, self.name) File "/home/ubuntu/miniconda3/lib/python3.7/site-packages/py4j/protocol.py", line 328, in get_return_value format(target_id, ".", name), value) py4j.protocol.Py4JJavaError: An error occurred while calling o0.sql. : java.lang.RuntimeException: org.verdictdb.exception.VerdictDBDbmsException: Issued the following query: create tabletpch1g.lineitem_xpartition by list columns (verdictdbblock) (partition p0 values in (0), partition p1 values in (1), partition p2 values in (2), partition p3 values in (3), partition p4 values in (4), partition p5 values in (5), partition p6 values in (6), partition p7 values in (7), partition p8 values in (8), partition p9 values in (9), partition p10 values in (10), partition p11 values in (11), partition p12 values in (12), partition p13 values in (13), partition p14 values in (14), partition p15 values in (15), partition p16 values in (16), partition p17 values in (17), partition p18 values in (18), partition p19 values in (19), partition p20 values in (20), partition p21 values in (21), partition p22 values in (22), partition p23 values in (23), partition p24 values in (24), partition p25 values in (25), partition p26 values in (26), partition p27 values in (27), partition p28 values in (28), partition p29 values in (29), partition p30 values in (30), partition p31 values in (31), partition p32 values in (32), partition p33 values in (33), partition p34 values in (34), partition p35 values in (35), partition p36 values in (36), partition p37 values in (37), partition p38 values in (38), partition p39 values in (39), partition p40 values in (40), partition p41 values in (41), partition p42 values in (42), partition p43 values in (43), partition p44 values in (44), partition p45 values in (45), partition p46 values in (46), partition p47 values in (47), partition p48 values in (48), partition p49 values in (49), partition p50 values in (50), partition p51 values in (51), partition p52 values in (52), partition p53 values in (53), partition p54 values in (54), partition p55 values in (55), partition p56 values in (56), partition p57 values in (57), partition p58 values in (58), partition p59 values in (59), partition p60 values in (60), partition p61 values in (61), partition p62 values in (62), partition p63 values in (63), partition p64 values in (64), partition p65 values in (65), partition p66 values in (66), partition p67 values in (67), partition p68 values in (68), partition p69 values in (69), partition p70 values in (70), partition p71 values in (71), partition p72 values in (72), partition p73 values in (73), partition p74 values in (74), partition p75 values in (75), partition p76 values in (76), partition p77 values in (77), partition p78 values in (78), partition p79 values in (79), partition p80 values in (80), partition p81 values in (81), partition p82 values in (82), partition p83 values in (83), partition p84 values in (84), partition p85 values in (85), partition p86 values in (86), partition p87 values in (87), partition p88 values in (88), partition p89 values in (89), partition p90 values in (90), partition p91 values in (91), partition p92 values in (92), partition p93 values in (93), partition p94 values in (94), partition p95 values in (95), partition p96 values in (96), partition p97 values in (97), partition p98 values in (98), partition p99 values in (99)) select t.l_orderkey, t.l_partkey, t.l_suppkey, t.l_linenumber, t.l_quantity, t.l_extendedprice, t.l_discount, t.l_tax, t.l_returnflag, t.l_linestatus, t.l_shipdate, t.l_commitdate, t.l_receiptdate, t.l_shipinstruct, t.l_shipmode, t.l_comment, t.l_dummy, 0 asverdictdbtier, cast(floor(rand() * 100) as unsigned) asverdictdbblockfromtpch1g.lineitemas t The table 'lineitem_x' is full at org.verdictdb.core.resulthandler.ExecutionTokenReader.takeOne(ExecutionTokenReader.java:53) at org.verdictdb.core.resulthandler.ExecutionTokenReader.next(ExecutionTokenReader.java:83) at org.verdictdb.core.execplan.ExecutablePlanRunner.runTillEnd(ExecutablePlanRunner.java:61) at org.verdictdb.coordinator.ScramblingCoordinator.scramble(ScramblingCoordinator.java:310) at org.verdictdb.coordinator.ScramblingCoordinator.scramble(ScramblingCoordinator.java:193) at org.verdictdb.coordinator.ExecutionContext.sql(ExecutionContext.java:171) at org.verdictdb.VerdictContext.sql(VerdictContext.java:388) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:282) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) Caused by: org.verdictdb.exception.VerdictDBDbmsException: Issued the following query: create tabletpch1g.lineitem_xpartition by list columns (verdictdbblock) (partition p0 values in (0), partition p1 values in (1), partition p2 values in (2), partition p3 values in (3), partition p4 values in (4), partition p5 values in (5), partition p6 values in (6), partition p7 values in (7), partition p8 values in (8), partition p9 values in (9), partition p10 values in (10), partition p11 values in (11), partition p12 values in (12), partition p13 values in (13), partition p14 values in (14), partition p15 values in (15), partition p16 values in (16), partition p17 values in (17), partition p18 values in (18), partition p19 values in (19), partition p20 values in (20), partition p21 values in (21), partition p22 values in (22), partition p23 values in (23), partition p24 values in (24), partition p25 values in (25), partition p26 values in (26), partition p27 values in (27), partition p28 values in (28), partition p29 values in (29), partition p30 values in (30), partition p31 values in (31), partition p32 values in (32), partition p33 values in (33), partition p34 values in (34), partition p35 values in (35), partition p36 values in (36), partition p37 values in (37), partition p38 values in (38), partition p39 values in (39), partition p40 values in (40), partition p41 values in (41), partition p42 values in (42), partition p43 values in (43), partition p44 values in (44), partition p45 values in (45), partition p46 values in (46), partition p47 values in (47), partition p48 values in (48), partition p49 values in (49), partition p50 values in (50), partition p51 values in (51), partition p52 values in (52), partition p53 values in (53), partition p54 values in (54), partition p55 values in (55), partition p56 values in (56), partition p57 values in (57), partition p58 values in (58), partition p59 values in (59), partition p60 values in (60), partition p61 values in (61), partition p62 values in (62), partition p63 values in (63), partition p64 values in (64), partition p65 values in (65), partition p66 values in (66), partition p67 values in (67), partition p68 values in (68), partition p69 values in (69), partition p70 values in (70), partition p71 values in (71), partition p72 values in (72), partition p73 values in (73), partition p74 values in (74), partition p75 values in (75), partition p76 values in (76), partition p77 values in (77), partition p78 values in (78), partition p79 values in (79), partition p80 values in (80), partition p81 values in (81), partition p82 values in (82), partition p83 values in (83), partition p84 values in (84), partition p85 values in (85), partition p86 values in (86), partition p87 values in (87), partition p88 values in (88), partition p89 values in (89), partition p90 values in (90), partition p91 values in (91), partition p92 values in (92), partition p93 values in (93), partition p94 values in (94), partition p95 values in (95), partition p96 values in (96), partition p97 values in (97), partition p98 values in (98), partition p99 values in (99)) select t.l_orderkey, t.l_partkey, t.l_suppkey, t.l_linenumber, t.l_quantity, t.l_extendedprice, t.l_discount, t.l_tax, t.l_returnflag, t.l_linestatus, t.l_shipdate, t.l_commitdate, t.l_receiptdate, t.l_shipinstruct, t.l_shipmode, t.l_comment, t.l_dummy, 0 asverdictdbtier, cast(floor(rand() * 100) as unsigned) asverdictdbblockfromtpch1g.lineitemas t The table 'lineitem_x' is full at org.verdictdb.connection.JdbcConnection.executeSingle(JdbcConnection.java:222) at org.verdictdb.connection.JdbcConnection.execute(JdbcConnection.java:182) at org.verdictdb.connection.CachedDbmsConnection.execute(CachedDbmsConnection.java:49) at org.verdictdb.core.execplan.ExecutableNodeRunner.execute(ExecutableNodeRunner.java:428) at org.verdictdb.core.execplan.ExecutableNodeRunner.executeAndBroadcast(ExecutableNodeRunner.java:411) at org.verdictdb.core.execplan.ExecutableNodeRunner.run(ExecutableNodeRunner.java:339) ... 1 more