[Bug]: load data report panic.

Ariznawlll commented 2 weeks ago

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Branch Name

main

Commit ID

eb1f0f578faab5fc7f977c23a3a99b9e62580186

Other Environment Information

- Hardware parameters:
- OS type:
- Others:

Actual Behavior

log： test.txt

Expected Behavior

load successfully

Steps to Reproduce

create database big_data_test;
use big_data_test;
CREATE TABLE `table_with_pk_index_for_load_100m` (
  `id` BIGINT NOT NULL,
  `col1` TINYINT DEFAULT NULL,
  `col2` SMALLINT DEFAULT NULL,
  `col3` INT DEFAULT NULL,
  `col4` BIGINT DEFAULT NULL,
  `col5` TINYINT UNSIGNED DEFAULT NULL,
  `col6` SMALLINT UNSIGNED DEFAULT NULL,
  `col7` INT UNSIGNED DEFAULT NULL,
  `col8` BIGINT UNSIGNED DEFAULT NULL,
  `col9` FLOAT DEFAULT NULL,
  `col10` DOUBLE DEFAULT NULL,
  `col11` VARCHAR(255) DEFAULT NULL,
  `col12` DATE DEFAULT NULL,
  `col13` DATETIME DEFAULT NULL,
  `col14` TIMESTAMP NULL DEFAULT NULL,
  `col15` BOOL DEFAULT NULL,
  `col16` DECIMAL(16,6) DEFAULT NULL,
  `col17` TEXT DEFAULT NULL,
  `col18` JSON DEFAULT NULL,
  `col19` BLOB DEFAULT NULL,
  `col20` BINARY(255) DEFAULT NULL,
  `col21` VARBINARY(255) DEFAULT NULL,
  `col22` VECF32(3) DEFAULT NULL,
  `col23` VECF32(3) DEFAULT NULL,
  `col24` VECF64(3) DEFAULT NULL,
  `col25` VECF64(3) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `col4` (`col4`),
  KEY `col3` (`col3`)
);

load data url s3option {'endpoint'='http://cos.ap-guangzhou.myqcloud.com','access_key_id'='','secret_access_key'='','bucket'='mo-load-guangzhou-1308875761','filepath'='mo-big-data/1000000000_20_columns_load_data_pk.csv'} into table big_data_test.table_with_pk_index_for_load_100M fields terminated by '|' lines terminated by '\n' ignore 99999999 lines parallel 'true';
（load sql please contact me）

Additional information

No response

jensenojs commented 1 week ago

表象是因为moCsvLineArray为空, 根本原因还需要细看一下

type ParseLineHandler struct {
    csvReader *csvparser.CSVParser
    // batch
    batchSize int
    // mo csv
    moCsvLineArray [][]csvparser.Field // wei nil
}

jensenojs commented 4 days ago

处理中移物联的insert pprof

jensenojs commented 1 day ago