stoneatom / stonedb

StoneDB is an Open-Source MySQL HTAP and MySQL-Native DataBase for OLTP, Real-Time Analytics, a counterpart of MySQLHeatWave. (https://stonedb.io)
https://stonedb.io/
GNU General Public License v2.0
862 stars 139 forks source link

feat: `Tianmu::core::GroupTable::Initialize(int64_t, bool): Assertion `distinct_size > 0'` failed. #1130

Open RingsC opened 1 year ago

RingsC commented 1 year ago

Have you read the Contributing Guidelines on issues?

Please confirm if bug report does NOT exists already ?

Describe the problem

./mysql-test-run.pl  --suite=tianmu --nowarnings --force --nocheck-testcases --retry=0 --parallel=10
[ 81%] tianmu.aggregate                         w3 [ fail ]
        Test ended at 2022-12-15 14:47:04

CURRENT_TEST: tianmu.aggregate
mysqltest: At line 86: query 'SELECT "count(distinct 17)_test1", count(distinct d1),count(distinct d2),count(distinct d3) FROM cs1' failed: 2013: Lost connection to MySQL server during query

The result from queries just before the failure was:
< snip >
(12345678901,123456.90128,0.1234567890123,2,1,1),
(NULL,NULL,NULL,2,1,1),
(0,0,0,2,1,1),
(1234,5678,0.12345678901234,3,2,2),
(1234567899,5678,0.1234567890123,3,2,2),
(12345678901,123456.90123,0.1234567890123,3,2,2),
(NULL,NULL,NULL,3,2,2),
(0,0,0,3,2,2);
SELECT "sum(17)_test1", sum(d1),sum(d2),sum(d3) FROM cs1;
sum(17)_test1   sum(d1) sum(d2) sum(d3)
sum(17)_test1   1222279007616   264347.8012300000   0.1234567890774000
SELECT "count(17)_test1", count(d1),count(d2),count(d3) FROM cs1;
count(17)_test1 count(d1)   count(d2)   count(d3)
count(17)_test1 11  11  11
SELECT "min(17)_test1", min(d1),min(d2),min(d3) FROM cs1;
min(17)_test1   min(d1) min(d2) min(d3)
min(17)_test1   -12345678923    -5678.0000000000    -0.1234567890123450
SELECT "max(17)_test1", max(d1),max(d2),max(d3) FROM cs1;
max(17)_test1   max(d1) max(d2) max(d3)
max(17)_test1   1234567890123   123856.9000000000   0.1234567890900000
safe_process[238190]: Child process: 238192, exit: 1

Server [mysqld.1 - pid: 236992, winpid: 236992, exit: 256] failed during test run
Server log from this test:
----------SERVER LOG START-----------
2022-12-15T06:46:33.495987Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2022-12-15T06:46:33.498292Z 0 [Warning] Insecure configuration for --secure-file-priv: Data directory is accessible through --secure-file-priv. Consider choosing a different directory.
2022-12-15T06:46:33.498307Z 0 [Warning] Insecure configuration for --secure-file-priv: Location is accessible to all OS users. Consider choosing a different directory.
2022-12-15T06:46:33.498345Z 0 [Note] /home/xxx/workshop/bin/bin/mysqld (mysqld 5.7.36-StoneDB-debug-log) starting as process 236994 ...
2022-12-15T06:46:33.507174Z 0 [Note] InnoDB: PUNCH HOLE support available
2022-12-15T06:46:33.507189Z 0 [Note] InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2022-12-15T06:46:33.507195Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2022-12-15T06:46:33.507199Z 0 [Note] InnoDB: Uses event mutexes
2022-12-15T06:46:33.507204Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2022-12-15T06:46:33.507209Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2022-12-15T06:46:33.507528Z 0 [Note] InnoDB: Number of pools: 1
2022-12-15T06:46:33.507628Z 0 [Note] InnoDB: Using CPU crc32 instructions
2022-12-15T06:46:33.512000Z 0 [Note] InnoDB: Initializing buffer pool, total size = 24M, instances = 1, chunk size = 24M
2022-12-15T06:46:33.531497Z 0 [Note] InnoDB: Completed initialization of buffer pool
2022-12-15T06:46:33.538949Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2022-12-15T06:46:33.550670Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2022-12-15T06:46:33.610691Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-12-15T06:46:33.610788Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-12-15T06:46:34.045271Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-12-15T06:46:34.055845Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2022-12-15T06:46:34.055871Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2022-12-15T06:46:34.057510Z 0 [Note] InnoDB: 5.7.36 started; log sequence number 1350784
2022-12-15T06:46:34.057698Z 0 [Note] InnoDB: Loading buffer pool(s) from /home/xxx/workshop/bin/mysql-test/var/3/mysqld.1/data/ib_buffer_pool
2022-12-15T06:46:34.058383Z 0 [Note] Plugin 'FEDERATED' is disabled.
2022-12-15T06:46:34.064899Z 0 [Note] InnoDB: Buffer pool(s) load completed at 221215  9:46:34
2022-12-15T06:46:34.464120Z 0 [Warning] unknown variable 'loose-mysqlx-port=13029'
2022-12-15T06:46:34.464143Z 0 [Warning] unknown variable 'loose-mysqlx-socket=/home/xxx/workshop/bin/mysql-test/var/tmp/3/mysqlx.1.sock'
2022-12-15T06:46:34.464752Z 0 [Note] Salting uuid generator variables, current_pid: 236994, server_start_time: 1671086793, bytes_sent: 0, 
2022-12-15T06:46:34.464810Z 0 [Note] Generated uuid: '373bed85-7c44-11ed-9777-000c29f02f1d', server_start_time: 11367648411107877003, bytes_sent: 94295925537552
2022-12-15T06:46:34.464866Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 373bed85-7c44-11ed-9777-000c29f02f1d.
2022-12-15T06:46:34.488750Z 0 [Note] Skipping generation of SSL certificates as options related to SSL are specified.
2022-12-15T06:46:34.488786Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
2022-12-15T06:46:34.488796Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
2022-12-15T06:46:34.489185Z 0 [Warning] CA certificate /home/xxx/workshop/bin/mysql-test/std_data/cacert.pem is self signed.
2022-12-15T06:46:34.489242Z 0 [Note] Skipping generation of RSA key pair as --sha256_password_auto_generate_rsa_keys is set to OFF.
2022-12-15T06:46:34.489281Z 0 [Warning] RSA private key file not found: /home/xxx/workshop/bin/mysql-test/var/3/mysqld.1/data//private_key.pem. Some authentication plugins will not work.
2022-12-15T06:46:34.489313Z 0 [Warning] RSA public key file not found: /home/xxx/workshop/bin/mysql-test/var/3/mysqld.1/data//public_key.pem. Some authentication plugins will not work.
2022-12-15T06:46:34.489332Z 0 [Note] Server hostname (bind-address): '*'; port: 13020
2022-12-15T06:46:34.496431Z 0 [Note] IPv6 is available.
2022-12-15T06:46:34.496493Z 0 [Note]   - '::' resolves to '::';
2022-12-15T06:46:34.496583Z 0 [Note] Server socket created on IP: '::'.
2022-12-15T06:46:34.700414Z 0 [Note] Failed to start slave threads for channel ''
2022-12-15T06:46:34.772301Z 0 [Note] Event Scheduler: Loaded 1 event
2022-12-15T06:46:34.772692Z 0 [Note] /home/xxx/workshop/bin/bin/mysqld: ready for connections.
Version: '5.7.36-StoneDB-debug-log'  socket: '/home/xxx/workshop/bin/mysql-test/var/tmp/3/mysqld.1.sock'  port: 13020  build-
mysqld: /home/xxx/workshop/stonedb-dev/storage/tianmu/core/group_table.cpp:371: void Tianmu::core::GroupTable::Initialize(int64_t, bool): Assertion `distinct_size > 0' failed.
06:46:37 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.

key_buffer_size=1048576
read_buffer_size=131072
max_used_connections=1
max_threads=151
thread_count=1
connection_count=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61094 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7f66bc000e10
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f696c04c618 thread_stack 0x40000
/home/xxx/workshop/bin/bin/mysqld(my_print_stacktrace+0x4b)[0x55c2f620f1f2]
/home/xxx/workshop/bin/bin/mysqld(handle_fatal_signal+0x3f7)[0x55c2f56fb624]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14420)[0x7f6974143420]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f6973e2f00b]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f6973e0e859]
/lib/x86_64-linux-gnu/libc.so.6(+0x22729)[0x7f6973e0e729]
/lib/x86_64-linux-gnu/libc.so.6(+0x33fd6)[0x7f6973e1ffd6]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core10GroupTable10InitializeElb+0x18df)[0x55c2f6ba7615]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core14GroupByWrapper10InitializeElb+0x37)[0x55c2f6bb4b8f]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core20AggregationAlgorithm9AggregateEbRlS2_PNS0_12ResultSenderE+0x93b)[0x55c2f6b464f9]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core9TempTable11MaterializeEbPNS0_12ResultSenderEb+0x96c)[0x55c2f687a824]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core6Engine7ExecuteEP3THDP3LEXP12Query_resultP18st_select_lex_unit+0xac4)[0x55c2f67fc47e]
/home/xxx/workshop/bin/bin/mysqld(_ZN6Tianmu4core6Engine12HandleSelectEP3THDP3LEXRP12Query_resultmRiS9_S9_i+0x957)[0x55c2f67fb0b1]
/home/xxxx/workshop/bin/bin/mysqld(_ZN6Tianmu7handler18ha_my_tianmu_queryEP3THDP3LEXRP12Query_resultmRiS8_S8_i+0x60)[0x55c2f6914fa2]
/home/xxx/workshop/bin/bin/mysqld(+0x2444862)[0x55c2f5e9a862]
/home/xxx/workshop/bin/bin/mysqld(_Z21mysql_execute_commandP3THDb+0xef8)[0x55c2f5e9388d]
/home/xxx/workshop/bin/bin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x648)[0x55c2f5e9b8f0]
/home/xxx/workshop/bin/bin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0xc48)[0x55c2f5e9043f]
/home/xxx/workshop/bin/bin/mysqld(_Z10do_commandP3THD+0x522)[0x55c2f5e8f2b6]
/home/xxx/workshop/bin/bin/mysqld(handle_connection+0x1f0)[0x55c2f5fd7bd8]
/home/xxx/workshop/bin/bin/mysqld(pfs_spawn_thread+0x174)[0x55c2f67213bb]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8609)[0x7f6974137609]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x43)[0x7f6973f0b133]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f66bc005b40): SELECT "count(distinct 17)_test1", count(distinct d1),count(distinct d2),count(distinct d3) FROM cs1
Connection ID (thread ID): 2
Status: NOT_KILLED

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Writing a core file
safe_process[236992]: Child process: 236994, killed by signal: 6

Expected behavior

No response

How To Reproduce

No response

Environment

No response

Are you interested in submitting a PR to solve the problem?

RingsC commented 1 year ago

assert failed as following.

void Tianmu::core::GroupTable::Initialize(int64_t, bool): Assertion `distinct_size > 0' failed.
  // pre-allocation of distinct memory
  int distinct_width = 0;
  int64_t distinct_size = 0;
  for (int i = no_grouping_attr; i < no_attr; i++)
    if (distinct[i]) {
      desc = aggregated_desc[i - no_grouping_attr];
      distinct_width = vc[i]->MaxStringSize() + 2;
      distinct_size += desc.max_no_values * distinct_width;
    }
  distinct_size = int64_t(ceil(distinct_size * 1.3));
  // initialize distinct tables
  for (int i = no_grouping_attr; i < no_attr; i++)
    if (distinct[i]) {
      desc = aggregated_desc[i - no_grouping_attr];
      DEBUG_ASSERT(distinct_size > 0);
      gdistinct[i] = std::make_shared<GroupDistinctTable>(p_power);
      gdistinct[i]->InitializeVC(vm_tab->RowNumberScope(), vc[i], desc.max_no_values,
                                 distinct_size / no_columns_with_distinct,
                                 (operation[i] != GT_Aggregation::GT_COUNT_NOT_NULL));  // otherwise must be decodable
      gdistinct[i]->CopyFromValueMatchingTable(vm_tab.get());
      distinct_size -= gdistinct[i]->BytesTaken();
      no_columns_with_distinct--;
    }

  // initialize everything
  Transaction *m_conn = current_txn_;

From the code, after initialization. we assume that distinct_size should be greater than 0. but, here, assert failed. it means that distinct_size maybe equals to 0.

Here, we can add one assertion statement to test distinct_size when and why it equals to 0.

if (distinct_size ==0) DEBUG_ASSERT(false);