Create and populate SQL database with current data

[Fehings]

Assuming we are going to wait for now to request the full data from the current claytonTV providers, we need to use the master spreadsheet to populate the database with as close to the real data as possible, so we have some data to use for testing.

Requirements:

• [x] Determine the fields that are required for the database,
• [x] Create corresponding Django models for the database
• [ ] Add a couple of data entries with data from the master spreadsheet to ensure we have some real videos in the database that we can use for testing embedding
• [ ] Populate the database with artificial dummy data (use a generator to make sure the data is similar in form to our real data)

[Ettie-ClaytonTV]

What data cleaning will need doing?

[Missive00]

What fields will need adding to the SQL database? Currently I understand that videos have a title + three tags (book of the bible, speaker, ministry area). Do we want to expand the number of tags to allow videos to be searched in more detail?

[Fehings]

NB we talked about using dummy data for this rather than faffing on extracting all the real data (as it is missing values compared to the actual clayton TV database which we will request at some point). Will change this issue accordingly.

[Fehings]

As for data cleaning, that will depend on the format we get the data in from the company behind the current site, hard to say now but I would guess it will involve reformatting into a set of csv files corresponding to tables in our database for easy import, and potentially some checks for missing data and coming up with protocols for how to deal with that.