Migrate TechTV Public Collections to OVS

[pdpinch]

This epic is part of a larger effort to shut down the TechTV web site.

There are three types of public collections on TechTV. Since they are similar, we can migrate them at the same time, applying slightly different business logic:

acceptance criteria

• [ ] 1. Public videos that cannot be uploaded to YouTube (e.g. due to lack of permission releases). These will only be available through Cloudfront. Example: MIT World
• [ ] 2. Public videos that are sensitive to region-blocking (e.g. must be viewable in China). These will be streamed through YouTube, with a failover to Cloudfront. Example: Blossoms, OCW
• [ ] 3. All public videos that don't fall into case 1 or 2 will be uploaded to YouTube. We won't stream these from CloudFront. Example: PTSG
• [ ] If captions exist in TechTV, they should be migrated to OVS

@emmello will provide a list of collections that fall into 1 and 2 above. All other collections will be handled as 3 above.

related issues

• Update Player to handle YouTube & Cloudfront streams

[mbertrand]

@pdpinch: should the original video still be copied over to S3 and made available for download to Dropbox (by admins), even though it will only play from Youtube? Each TechTV video typically has the original uploaded file plus the transcoded MP4 video files.

[mbertrand]

Region-blocking collection ids: 575,587,1081,1214,2191,2884,2885,2886,2887,2888,2890,2891,2894,2956,3152,3251,3254,3255,3256,3257,3326,3384,3401

Not on Youtube: 2228,3672

[pdpinch]

I believe that was our plan.

As a separate issue, I'd still like to develop a lifecycle for these public videos where, when they aren't viewed, they are cycled off S3.

should the original video still be copied over to S3 and made available for download to Dropbox

[mbertrand]

There are about 10,500 public videos. The OVS-QA Youtube account has a standard API quota of 1,000,000 "units"/day, and according to the docs a video upload uses on average 1600 units, so that's a max of about 625 uploads/day. Unless we apply to increase that we may have to batch imports over multiple days.

[mbertrand]

We can run the import script on certain collections at a time via the --collections command-line parameter: --collections 575 587 2191

Another idea is to just spread out the YouTube uploads over multiple days, though this will conflict a bit with acceptance criterion # 3 in the description:

• Modify the Import script to not upload anything to YouTube (and not create any associated YoutubeVideo objects)
• Create a daily celery task that will grab the first few hundred public videos that do not yet have an associated YoutubeVideo object and upload them to YouTube (except for videos matching criterion # 1).
• All public videos will play from Cloudfront if no YoutubeVideo object exists for it yet. If a YoutubeVideo object exists for the video, then criterion 3 above will apply.