search

nmadhire / jwpl

Automatically exported from code.google.com/p/jwpl
0 stars 0 forks source link

[RevisionMachine] DiffTool produces "INSERT INTO revisisions VALUES;" without values #27

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
When processing the English Wikipedia dump 20110405, the DiffTool produced the 
following insert statement 
    INSERT INTO revisions VALUES;
without giving any values. 

It was impossible to import this sql file to the database. The line had to be 
removed.

It should be ensured that only legal SQL statements are produced.

Original issue reported on code.google.com by oliver.ferschke on 11 Jun 2011 at 9:55

GoogleCodeExporter commented 9 years ago

Original comment by oliver.ferschke on 11 Jun 2011 at 9:56

GoogleCodeExporter commented 9 years ago 
Workaround for this problem, until it is fixed:

So far, the problem only occured ONCE with the English WP dump 20110405.
When mysqlimport reports the error, it is possible to delete the corrupt line 
from the dump and import the fixed dump without problems.

If, for example, mysqlimport reports an error in line 174633037 of the dump, 
you can display this line using
awk 'FNR == 174633037 { print; exit }' output_1.sql 

If the line only contains 
INSERT INTO revisions VALUES;
without any actual values, it must be removed. This can be achieved with 
sed '174633037d' output_1.sql > output_1.sql.fixed

The above examples work with decompressed dump files. For compressed dumps, you 
will have to add bzip to the command line pipe.

Original comment by oliver.ferschke on 14 Aug 2011 at 12:18

GoogleCodeExporter commented 9 years ago 
The problems are caused by SQLEncoder.encodeTask(Task) and 
SQLEncoder.binaryTask(Task)

Original comment by oliver.ferschke on 20 Sep 2011 at 8:57

GoogleCodeExporter commented 9 years ago

Original comment by oliver.ferschke on 7 Oct 2011 at 10:55

GoogleCodeExporter commented 9 years ago 
This issue should be solved now. However, it has not been tested on a large 
dump of the English Wikipedia. If it fails again in the future, reopen bug.

Original comment by oliver.ferschke on 8 Nov 2011 at 10:19

GoogleCodeExporter commented 9 years ago

Original comment by oliver.ferschke on 16 Feb 2012 at 1:19