Error finding mods_0.txt

[Superchicken]

Ubuntu 16.10 MySQL5.7

ERR: SQLException in main.cpp (print_sql_error) on line 215 ERR: Can't get stat of '/home/mysql/POE/mods_0.txt' (Errcode: 2 - No such file or directory) (MySQL error code: 13 , SQLState: HY000 ) LOAD DATA CONCURRENT INFILE 'mods_0.txt' REPLACE INTO TABLE Mods FIELDS TERMINATED BY ',' ENCLOSED BY '"' ESCAPED BY '"' LINES TERMINATED BY ' '

Before the error occurs, MySQL is executing query LOAD DATA CONCURRENT INFILE 'mods_0.txt'... and it seems like the moment the query completes the indexer throws the error. I think this is the very first 'LOAD DATA...' for mods on an empty table and it completes successfully (130k rows added, ~400s). The mod queue length is 2.

After the error my DB_DATA_DIR contains mods_1.txt and no mods_0.txt

Let me know what other info would be useful.

[Superchicken]

I think I've narrowed down the issue to two things, it looks like the it applies to each insertion thread.

New files are pushed and popped at the front of the queue (LIFO)

This doesn't actually break things, but it makes sense to me that one would want to sequentially process the river if it starts giving you more data than you can handle.

main.cpp line 567 mod_queue.push_front( replace_string( path.str(), DB_DATA_DIR + "/", "" ));

Files are named in a way that create conflicts.

If there are 2 files in the queue (mod_0, mod_1), when one is flushed (now only mod_1 remains) attempting to create a new file would make a file named mod_1

main.cpp line 555 path << DB_DATADIR << "/mods" << mod_queue.size() << ".txt";

The first one is fixed easily

modqueue.pushfrontback( replace_string( path.str(), DB_DATA_DIR + "/", "" ));

The second one I've put a band-aid on and am not too pleased with

path << DB_DATADIR << "/mods" << mod_queue.size()random_id(10) << ".txt";

It's possible to get a collision on the filenames but I'm gambling that it won't happen.