DB bulk insert시 확인하면 좋을 것들

[ecsimsw]

https://dev.mysql.com/doc/refman/8.0/en/load-data.html

https://aws.amazon.com/ko/blogs/database/improve-performance-of-your-bulk-data-import-to-amazon-rds-for-mysql/

[ecsimsw]

• Use INSERT statements with multiple values
• Batching flat files with LOAD DATA
• Disable autocommit
• Increase the size of innodb_log_buffer_size
• Disable foreign key and unique key checks

[ecsimsw]

LOAD DATA INFILE "/var/lib/mysql-files/init-data-1_000_000.csv" INTO TABLE product FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 ROWS;

4개 컬럼 (long, char, int, int) 100만개 : 3초 1억개 : 1시간 45분