search

stoneatom / stonedb

StoneDB is an Open-Source MySQL HTAP and MySQL-Native DataBase for OLTP, Real-Time Analytics, a counterpart of MySQLHeatWave. (https://stonedb.io)
https://stonedb.io/
GNU General Public License v2.0
862 stars 139 forks source link

bug: load data problem #1465

Open davidshiz opened 1 year ago

davidshiz commented 1 year ago

Have you read the Contributing Guidelines on issues?

Please confirm if bug report does NOT exists already ?

Describe the problem

lineitem table, 59986052 rows of data，after load data，The number of data rows returned is incorrect

root@ub01:/data/data_10g_tpch# wc -l lineitem.tbl
59986052 lineitem.tbl
mysql> load data local infile '/data/data_10g_tpch/lineitem.tbl'  into table lineitem fields terminated by '|';
Query OK, 59986055 rows affected, 93 warnings (11 min 3.18 sec)
Records: 59986055  Deleted: 0  Skipped: 0  Warnings: 93

The first three rows of result set below are abnormal

mysql> select * from lineitem order by 1 asc limit 10;
+------------+-----------+-----------+--------------+------------+-----------------+------------+-------+--------------+--------------+------------+--------------+---------------+-------------------+------------+------------------------------------+
| L_ORDERKEY | L_PARTKEY | L_SUPPKEY | L_LINENUMBER | L_QUANTITY | L_EXTENDEDPRICE | L_DISCOUNT | L_TAX | L_RETURNFLAG | L_LINESTATUS | L_SHIPDATE | L_COMMITDATE | L_RECEIPTDATE | L_SHIPINSTRUCT    | L_SHIPMODE | L_COMMENT                          |
+------------+-----------+-----------+--------------+------------+-----------------+------------+-------+--------------+--------------+------------+--------------+---------------+-------------------+------------+------------------------------------+
|          0 |         0 |         0 |            0 |       0.00 |            0.00 |       0.00 |  0.00 |              |              | 0000-00-00 | 0000-00-00   | 0000-00-00    |                   |            |                                    |
|          0 |         0 |         0 |            0 |       0.00 |            0.00 |       0.00 |  0.00 |              |              | 0000-00-00 | 0000-00-00   | 0000-00-00    |                   |            |                                    |
|          0 |         0 |         0 |            0 |       0.00 |            0.00 |       0.00 |  0.00 |              |              | 0000-00-00 | 0000-00-00   | 0000-00-00    |                   |            |                                    |
|          1 |    240267 |     15274 |            5 |      24.00 |        28974.00 |       0.10 |  0.04 | N            | O            | 1996-03-30 | 1996-03-14   | 1996-04-01    | NONE              | FOB        |  pending foxes. slyly re           |
|          1 |   1551894 |     76910 |            1 |      17.00 |        33078.94 |       0.04 |  0.02 | N            | O            | 1996-03-13 | 1996-02-12   | 1996-03-22    | DELIVER IN PERSON | TRUCK      | egular courts above the            |
|          1 |    673091 |     73092 |            2 |      36.00 |        38306.16 |       0.09 |  0.06 | N            | O            | 1996-04-12 | 1996-02-28   | 1996-04-20    | TAKE BACK RETURN  | MAIL       | ly final dependencies: slyly bold  |
|          1 |    156345 |      6348 |            6 |      32.00 |        44842.88 |       0.07 |  0.02 | N            | O            | 1996-01-30 | 1996-02-07   | 1996-02-03    | DELIVER IN PERSON | MAIL       | arefully slyly ex                  |
|          1 |    636998 |     36999 |            3 |       8.00 |        15479.68 |       0.10 |  0.02 | N            | O            | 1996-01-29 | 1996-03-05   | 1996-01-31    | TAKE BACK RETURN  | REG AIR    | riously. regular, express dep      |
|          1 |     21315 |     46316 |            4 |      28.00 |        34616.68 |       0.09 |  0.06 | N            | O            | 1996-04-21 | 1996-03-30   | 1996-05-16    | NONE              | AIR        | lites. fluffily even de            |
|          2 |   1061698 |     11719 |            1 |      38.00 |        63066.32 |       0.00 |  0.05 | N            | O            | 1997-01-28 | 1997-01-14   | 1997-02-02    | TAKE BACK RETURN  | RAIL       | ven requests. deposits breach a    |
+------------+-----------+-----------+--------------+------------+-----------------+------------+-------+--------------+--------------+------------+--------------+---------------+-------------------+------------+------------------------------------+
10 rows in set (0.57 sec)

Expected behavior

No response

How To Reproduce

No response

Environment

root@ub01:/data/data_10g_tpch# /stonedb57/install/bin/mysqld --version
/stonedb57/install/bin/mysqld  Ver 5.7.36-StoneDB-v1.0.3 for Linux on x86_64 (build-)
build information as follow:
        Repository address: https://github.com/stoneatom/stonedb.git:stonedb-5.7-dev
        Branch name: stonedb-5.7-dev
        Last commit ID: 0a2f9be67
        Last commit time: Date:   Mon Mar 20 20:32:42 2023 +0800
        Build time: Date: Tue Mar 21 16:32:45 CST 2023

Are you interested in submitting a PR to solve the problem?

davidshiz commented 1 year ago 
mysql> show warnings;
+---------+------+-------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                         |
+---------+------+-------------------------------------------------------------------------------------------------+
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 23349658 doesn't contain data for all columns                                               |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_ORDERKEY' at row 23349658      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_PARTKEY' at row 23349658       |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SUPPKEY' at row 23349658       |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_LINENUMBER' at row 23349658    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_QUANTITY' at row 23349658      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_EXTENDEDPRICE' at row 23349658 |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_DISCOUNT' at row 23349658      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_TAX' at row 23349658           |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_RETURNFLAG' at row 23349658    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_LINESTATUS' at row 23349658    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPDATE' at row 23349658      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_COMMITDATE' at row 23349658    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_RECEIPTDATE' at row 23349658   |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPINSTRUCT' at row 23349658  |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPMODE' at row 23349658      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_COMMENT' at row 23349658       |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 38858720 doesn't contain data for all columns                                               |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_ORDERKEY' at row 38858720      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_PARTKEY' at row 38858720       |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SUPPKEY' at row 38858720       |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_LINENUMBER' at row 38858720    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_QUANTITY' at row 38858720      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_EXTENDEDPRICE' at row 38858720 |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_DISCOUNT' at row 38858720      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_TAX' at row 38858720           |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_RETURNFLAG' at row 38858720    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_LINESTATUS' at row 38858720    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPDATE' at row 38858720      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_COMMITDATE' at row 38858720    |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_RECEIPTDATE' at row 38858720   |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPINSTRUCT' at row 38858720  |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_SHIPMODE' at row 38858720      |
| Warning | 1263 | Column set to default value; NULL supplied to NOT NULL column 'L_COMMENT' at row 38858720       |
| Warning | 1261 | Row 39892758 doesn't contain data for all columns                                               |
| Warning | 1261 | Row 39892758 doesn't contain data for all columns                                               |
+---------+------+-------------------------------------------------------------------------------------------------+
64 rows in set (0.00 sec)