search

dungates / NBAnalytics

Statistics, Visualization, and Machine Learning with NBA Data in R
0 stars 0 forks source link

Importing data onto RDS MySQL #2

Open cwickham opened 3 years ago

cwickham commented 3 years ago

These are the rough steps:

  1. Set up a new MySQL database, and create two tables one for movement data, one for play by play data.
  2. Get JSON onto the RDS server
  3. Import JSON into the MySQL database

Step 2. One option is to transfer your local files to server (not recommended, becasue it will be limited by your upload speed). Other option, directly get data from website to your server - this needs you to translate whatever you did (point-click-select) to code of some sort.

It might be Step 2 & 3 can happen simultaneously from MySQL Workbench.

To get started

Start small: start with play by play data for one game. Start with CSV. You might just start with some columns.

Goal:

  1. Try working through the three (two?) steps above. You'll have to hunt for resources. Maybe for Step 2 & 3 check out https://www.databasestar.com/mysql-workbench-import-csv/
  2. Connect to it from your local R, and look at the data. Start at: https://dbplyr.tidyverse.org/#usage

Only then, think about scaling up to all play-by-play data.

Then, figure out how to get JSON data in MySQL. (Charlotte will look around for that).

cwickham commented 3 years ago

Ideas for Getting JSON files to RDS server

Github with all zipped JSONs: https://github.com/rajshah4/BasketballData/raw/master/2016.NBA.Raw.SportVU.Game.Logs/01.01.2016.CHA.at.TOR.7z One strategy might be to get download links i.e (https://github.com/rajshah4/BasketballData/raw/master/2016.NBA.Raw.SportVU.Game.Logs/01.01.2016.CHA.at.TOR.7z) and then have curl? download them direct to RDS server.