LOAD DATA INFILE block on reading file

[huangminwen]

I took the part of the code from the project, https://github.com/huangminwen/load-file-test, it only modify the database config in the yml file, I try 3 method on this test

load file way
by stream way
batch insert way

when you run this test code, please exeute the sql file, and the put the eg_nat202109-1632813678309-588228.tmp file on /opt/inap/filedata/audit/loadfile path, you can put it anywhere actually, but should modify the code com.study.storage.NatLogSaveThread#FILE_PATH at the same time.

It's about 15mins you can see the block on reading file on database with SELECT * FROM information_schema.PROCESSLIST ORDER BY time ASC;

I try other version, such as 10.2、10.2.40、10.3.31 even MySQL 8, them work well.

note: I use data base with docker redirectly.

Is this a bug in version 10.2.36?

more information please see https://stackoverflow.com/questions/69425370/mariadb-load-data-infile-block-on-reading-file

Thanks in advance.

[grooverdan]

For those not familar with java Spring applications, how exactly do you run in these 3 modes? I did get the build with mvn package.

If you've identified that 10.2.40 is unaffected, why do you need to determine if 10.2.36 has a bug? What decision are you going to make if it is a bug compared it if it isn't?

[huangminwen]

For those not familar with java Spring applications, how exactly do you run in these 3 modes? I did get the build with mvn package.

If you've identified that 10.2.40 is unaffected, why do you need to determine if 10.2.36 has a bug? What decision are you going to make if it is a bug compared it if it isn't?

by modify com.study.task.LogStart#run method, like that

runThreads("NatLogPool", 5, NatLogSaveThread.class);

and

runThreads("NatLogPool", 5, NatLogSaveThreadByBatch.class);

and

runThreads("NatLogPool", 5, NatLogSaveThreadByStream.class);

because my team use version 10.2.36 now, it's slow to lead to process the data and then affect performance, so wo will decision upgrade the version, but not sure whether the bug is in version 10.2.36, if it is a bug, we should upgrade the version.

If you try the code, and then see the block on reading file on database.

note: this test code is work well in 10.2、10.2.40、10.3.31 even MySQL 8.

[grooverdan]

10.2 is just an alias for 10.2.40.

You have the test case to prove its a bug, to your team. You can also prove it to be a bug by starting at the previous versions before 10.2.36, showing the version that degrades, and the version after 10.2.36 that improves.

Pinning a commit or bug number that introduced the regression is a lot of effort, and shouldn't add much more weight to the testing above.

So its good to have this as a test case that represents your workload. A good set of these will prove valuable in testing upgrades.

While I can help with outstanding issues, you have all you need to complete the upgrade. My assistance is not required.