Ornias1993 / fetlife-aslsearch-reborn

Tampermonkey user script offering an interface to perform pseudo-automatic searches of the FetLife.com user base filtered by age, sex, location, and role.

6 stars 2 forks source link

Design, Test and Document a workflow to migrate google docs databases #61

Open Ornias1993 opened 5 years ago

Ornias1993 commented 5 years ago

Description

With the transfer to SQL also comes the need to transfer the current database. Although we currently have a 4,5 milion entry database within SQL, the resolution (number of colums filled) is a lot higher and it's a lot more up-to-date with the freshly scraped data

Category

TODO

Ornias1993 commented 5 years ago

In progress, made preliminary tool to create a CSV (needs some love though) from all old xlsx files. CSV needs some tweaking to be more easily importable into MYSQL.

Workflow would be pretty much:

install python}
pip the imports
Run file
Copy to sql server
Run import on SQL server

Ornias1993 commented 5 years ago

okey, got some importing going. Not great, but it does the trick. I doubt its worth the time making it beter. Just needs some cleanup, comments and fix a bug where some of the "sql file" import commands are not placed right.