Sync transcripts for all videos

[nedtwigg]

We'll do these on feature/syncTranscripts. If it's helpful to merge this into the branch currently under active development, it's always easy to do so.

When you start transcribing a video, check its box to make sure that nobody else starts duplicating your work. For info on how to do this, see foundation-gen/TRANSCRIPT_SYNC_HOW_TO.md.

1960 - Kennedy, Nixon

• [x] 1960-09-26
• [x] 1960-10-07
• [ ] 1960-10-13
• [x] 1960-10-21

1976 - Carter, Ford

• [x] 1976-09-23
• [ ] 1976-10-06
• [ ] 1976-10-22

1980 - Anderson, Carter, Reagan

• [ ] 1980-09-21
• [x] 1980-10-28

1984 - Mondale, Reagan

• [x] 1984-10-07
• [ ] 1984-10-21

1988 - Bush, Dukakis

• [ ] 1988-09-25
• [ ] 1988-10-13

1992 - Bush, Clinton, Perot

• [ ] 1992-10-11
• [ ] 1992-10-15
• [x] 1992-10-19

1996 - Clinton, Dole

• [x] 1996-10-06
• [ ] 1996-10-16

2000 - Bush, Gore

• [ ] 2000-10-03
• [ ] 2000-10-11
• [x] 2000-10-17

2004 - Bush, Kerry

• [ ] 2004-09-30
• [ ] 2004-10-08
• [ ] 2004-10-13

2008 - McCain, Obama

• [ ] 2008-09-26
• [ ] 2008-10-07
• [ ] 2008-10-13

2012 - Obama, Romney

• [ ] 2012-10-03
• [ ] 2012-10-16
• [ ] 2012-10-22

2016 - Clinton, Trump

• [ ] 2016-09-26
• [ ] 2016-10-09
• [ ] 2016-10-19

[WebsByTodd]

I started on Obama-Romney 1 and ran into a few issues:

• At 3027 seconds there is a sentence or two spoken by Obama in the transcript that was cut out of the video. Also most of Obama's first turn is muted. Might need to replace the video.
• At the current commit, 94e7de6, there is a null pointer exception that I can't figure out.
• Ellipses (...) don't play nicely with the tool.
• This transcript is broken up into paragraphs on debates.org. I think the tool should recognize when a line doesn't start with a speaker and auto-assign it to the previous speaker.
• This transcript also sometimes has the same speaker appearing multiple times in a row. The tool should probably condense these into a single turn too. Or maybe this particular case needs further discussion.

[WebsByTodd]

More notes:

• I'm seeing the same null pointer exception with Obama/McCain 3 highlight 1.
• The Obama/McCain 3 video hangs on highlight 2 and won't load. It works fine in a web browser though.
• Double hyphens (--) don't play nicely with the tool.
• The date was wrong on the 3rd Obama/McCain video. Was 10/13, not it's 10/15.

[nedtwigg]

I fixed the exceptions you found. Regarding punctuation:

• self-correcting - treated as one word
• did - did - did I stutter? - correct
• did-did-did I stutter? - incorrect, did-did-did is one big word
• did... did... did I stutter? - incorrect

Words should always be separated by spaces, and the only punctuation which is allowed to not touch a word is -.

I added Spotless rules to enforce this, and updated for the transcripts that were already in there. We can use it to make sure all the transcripts are consistent.

[WebsByTodd]

Notes for Obama/Romney 1

• No video/audio from 4852-4857 in the YouTube video.
• Any thoughts on "Don't ask/don't tell"? Formatting this phrase could be important for the search feature.
  • There is 1 final unmatched result that throws a null pointer exception.

[WebsByTodd]

I noticed a trend toward middle initials and I vaguely remember a conversation. Is there a guideline to follow here? What about for folks like Bill Clinton (William J Clinton) and Bob Dole (Robert J Dole) and Mitt Romney (Willard M Romney) whose full names might be less recognizable than the names they are known by?

--edit - I've updated NAME_AND_TITLE_DESIGN.md with this info.

[nedtwigg]

The checkmarks above were all formally synced. In addition, the following had been synced at one time (which means the .said is very reliable), but they need to be-synced against their new sources. It is possible there are problems in the .vtt for these new sources:

• Bush, Kerry (3 of 3)
• McCain, Obama (2 of 3)
• Obama, Romney (1 of 3)
• Clinton, Trump (2 of 3)

And for all of the rest, the .said was taken straight from debates.org, and autosynced by SetVttToSaid.java in #269. In the future, we will need to revisit these.