licoffe / POE-Stash-indexer-NG

High performance Path of Exile stash indexer
MIT License
43 stars 12 forks source link

Data truncated for column 'modValue1' #3

Closed Superchicken closed 7 years ago

Superchicken commented 7 years ago

Running on Ubuntu 16.10 mysql 5.7.

I'm receiving error:

ERR: SQLException in main.cpp (print_sql_error) on line 215 ERR: Data truncated for column 'modValue1' at row 19 (MySQL error code: 1265 , SQLState: 01000 ) LOAD DATA CONCURRENT INFILE 'mods_0.txt' REPLACE INTO TABLE Mods FIELDS TERMINATED BY ',' >ENCLOSED BY '"' ESCAPED BY '"' LINES TERMINATED BY ' '

Looking at the line causing the error, I notice it is the first line to have no value specified for modValue1:

"5e968f419af74b6c673927fd0f083ab3599573d2879d1f6e9c27085a761ea9fa","Immune to Curses during Flask effect;Removes Curses on use","","","","","EXPLICIT","5e968f419af74b6c673927fd0f083ab3599573d2879d1f6e9c27085a761ea9fa2"

Manually editing in a "0" for modValue1 and manually executing the SQL query resolves the error for that line, but pushes the error to the next instance of a blank valued modValue1.

It seems like LOAD DATA INFILE isn't handling empty values correctly, source

Superchicken commented 7 years ago

My current workaround is changing the SQL query on line 1316 in main.cpp

threaded_insert( "LOAD DATA CONCURRENT INFILE '" + mod_file + "' REPLACE INTO TABLE Mods FIELDS TERMINATED BY ',' ENCLOSED BY '\"' ESCAPED BY '\"' LINES TERMINATED BY '\n' (itemId, modName, @vmodValue1, modValue2, modValue3, modValue4, modType, modKey) set modValue1 = nullif(@vmodValue1, '')" );

Where my additions are in bold. I'm not sure how much this will affect performance or if it's a suitable solution.

licoffe commented 7 years ago

I modified the schema.sql, it seems to fix the issue. Could you try importing it and run the indexer again?

Superchicken commented 7 years ago

Thanks, your changes have resolved this issue.

licoffe commented 7 years ago

Sure, glad it helped :)