[#15] Bash script to migrate factoids, karma, and logs from Drupal.

[eojthebrave]

This bash script will import karma, factoids, and logs from a Drupal bot into lubot. It does so by directly exporting data from MySQL into a CSV file and then using the mongoimport command. It's pretty fast. Though logs can take a while. The #lullabot logs DB has about 1.5 million records in it. A few records had errors, but it completed eventually. Generally for testing, I would edit the script and add a LIMIT clause to the SQL query for the bot_log table.

Resolves #15 - Or at least as long as all you want to import is karma, factoids, and logs. Though the script can be easily edited for other data import/export as needed.

Command

The command drupal-migrate.sh takes three arguments.

-m = arguments to pass to the mysql command line tool in order to run our export query. User must have SELECT INTO permissions.

-d = name of the monogdb DB to use when importing

-c = IRC channel to use for imported records. Some Drupal tables don't track the channel a record belongs to, but lubot requires this for all data. So we need to provide a default.

Example

Command: ./drupal-migrate.sh -m "-u root -p -h localhost lullabot_irc_bot" -d lubot -c "#lullabot"

Output:

Exporting karma data from MySQL
Enter password:
Importing karma data
connected to: 127.0.0.1
imported 1931 objects
Exporting factoid data from MySQL
Enter password:
Importing factoid data
connected to: 127.0.0.1
imported 641 objects
Exporting logs data from MySQL
Enter password:
Importing log data
connected to: 127.0.0.1
imported 1800 objects
SUCCESS!

[eojthebrave]

I suppose if we don't want this to be in the lubot repo itself as it is Drupal specific, but lubot is not ... let me know I can added it to it's own repo or something like that.

[q0rban]

Nice work, Joe! I haven't tested this out yet, but it looks good.