blaze / odo

Data Migration for the Blaze Project
http://odo.readthedocs.org/
BSD 3-Clause "New" or "Revised" License
1.01k stars 138 forks source link

Support "LOAD DATA LOCAL INFILE" in MySql #591

Open haihongwang opened 7 years ago

haihongwang commented 7 years ago

I am trying to load a local csv file to a remote MySql server using odo lib, but I got the following error:

InternalError: (pymysql.err.InternalError) (1290, u'The MySQL server is running with the --secure-file-priv option so it cannot execute this statement') [SQL: u'LOAD DATA INFILE %(path)s\n INTO TABLE GlobalObjects\n CHARACTER SET %(encoding)s\n FIELDS\n TERMINATED BY %(delimiter)s\n ENCLOSED BY %(quotechar)s\n ESCAPED BY %(escapechar)s\n LINES TERMINATED BY %(lineterminator)s\n IGNORE %(skiprows)s LINES\n '] [parameters: {'lineterminator': '\n', 'encoding': 'utf8', 'skiprows': 1, 'delimiter': ',', 'escapechar': '\', 'quotechar': u'"', 'path': '/tmp/my_table.csv'}]

It does not appear it use "LOCAL DATA LOCAL INFILE" statement, which is wrong for loading local file to remote server.

steezeburger commented 6 years ago

https://github.com/blaze/odo/blob/master/odo/backends/sql_csv.py#L127

It seems there's a way to define LOCAL in this code, but I was not able to figure out how that method was called to see what theelement was to set local to true.

jaygorrell commented 6 years ago

@necaris helped me find that you can simply pass the value of local to odo(). The value literally gets dropped in, so you'd just use local='LOCAL'.