Complete Twitter Bot

[walshbr]

Peeling this off as the technical component of #402. The following tasks are included in finishing the bot out.

• [x] Include Twitter usernames of authors in the tweet contents.
• [x] Sanitize credentials in the script so that they are stored in Heroku.
• [x] Shift to Google spreadsheet rather than CSV for data storage.
• [x] Post bot in version controlled repo.
• [x] Document the script for the bot better.
• [x] Update editorial documentation to allow for adding new lessons to the spreadsheet.
• [x] Incorporate testing/error reporting so we don't accidentally tweet nonsense when data is formatted incorrectly.
• [x] Refine the algorithm so it will cycle through all possible lessons before repeating.
• [x] Shift bot to official Programming Historian Twitter credentials.

[mdlincoln]

@walshbr When ready, will you post this bot code in a repo within the programminghistorian team? (I think that would be great!)

[walshbr]

@mdlincoln for sure! I ran into issues sanitizing the credentials - the Twitter stuff was hidden pretty easily, but Google Spreadsheets is calling for a JSON key. Hiding the contents of the Google key in the environment variables on Heroku was making things wonky, so I took a break from sanitizing it to do some other debugging and fleshing out. I'll take a look again and see if I can get the credentials to be hidden. I'll post it up here once I can find a workaround.

Right now I've got it tweeting out twice a week from a Google spreadsheet, once on Monday and once on Thursday. Hopefully these times account for as much of the community as possible:

• Monday - 9AM PST/12PM EST/4PM UTC
• Thursday 6:30 AM PST/9:30 AM EST/1:30PM UTC

I think the Spanish tweets can just be a second sheet within the larger one, so the Spanish team can just fill out the sheet at their leisure and let me know when they're ready to flip the switch on it. I'll update with more soon.

[mdlincoln]

Good point! I've no idea if there's any way to do that for google sheets. I suppose you could always just keep the JSON key untracked, and then any future maintainer will have to generate their own key for the sheet (if I understand how that's working...?)

[walshbr]

Could work for us each to have our generated key for the sheet, but we'd have to go through the process of generate the key and adding that api account as a shared account on the sheet. Seems like a bit of a pain.

I think I worked out another way - basically the same thing except a shared key. The workflow feels a bit wonky, but sounds similar to what you're suggesting. I have the master branch pushing to a repository without tracking the JSON key. And then I have a separate heroku branch that I just never synch up to github. This branch exists locally and does have the JSON key on it. So I work on master and push up to github to keep things in sync, merge things over to the heroku branch locally, and then push that branch up to heroku with the key. Anyone else should be able to replicate it, so long as we share the JSON key by email and manually add it to the local heroku branch the first time. So I suppose it's a question of whether there is a shared key to be passed around or a separate key to be generated, added locally, and then edited inline in the script. One seems more secure, one seems easier.

I've got a more verbose writeup with the commands at the repository here that I've now pushed. Let me know if that workflow doesn't seem sustainable or doesn't make sense (though theoretically I guess there won't be a lot of ongoing develop on this). I can move the repository over whenever, though I'm also happy to leave it over there now until I'm done testing and debugging. Still got a few other things to work out.

[walshbr]

Ah and FWIW at the moment it's working with a pretty narrow band of lessons for testing so as not to spam the authors of those lessons until it's ready for primetime. Just need to fill in the rest of the sheet and then take the brakes off.

[walshbr]

FYI - all looks to be working. I'm working through the process of adding to the contents of the twitter sheet to fill things out before we go live.

[walshbr]

@mdlincoln - this is about ready for primetime. The tweets dataset is all filled out - I just want to do one last pass over them to make they're all 140 characters. Though anything like that should just get caught in the logs anyway and not make it onto the account. Looks like I need to be given permission to create repositories for the programminghistorian organization in order to transfer ownership. Or I could just give the bot to you and then have you move it to the PH org?

I'll also need to register the Programming Historian Twitter account with the Twitter API to transfer the scripted tweets over to those credentials rather than the development account I've been using. @acrymble - should I go ahead and do that?

Happy to discuss more first if people want, and also happy to have people take a look at the tweet sheet if they want to edit anything before the bot starts going through our real credentials. It's a slow burn, though, as the bot only using one row in the sheet per week. So should be pretty easy to edit things as we go if people have issues with anything.

The bot will need just a bit more updating to account for Spanish translations/tweets when the Spanish team is set to go live with them. We'll store those in a separate tab within the main spreadsheet, I think, and right now it's only set to read in from the one tab.

[mdlincoln]

Looks like I need to be given permission to create repositories for the programminghistorian organization in order to transfer ownership. Or I could just give the bot to you and then have you move it to the PH org?

An "owner" of the ph org (e.g. @wcaleb @fredgibbs @acrymble @ahegel et al) needs to do that. (Are half the team supposed to be members and half owners? What's going on there?)

[ianmilligan1]

I just noticed that the other day, @mdlincoln, and I figure it's just a legacy issue. I probably don't need to be an owner and could just be a member, but others like you definitely do. I guess I'll leave it to @wcaleb to make the changes? Or I can?

[wcaleb]

I've updated everyone's permissions to make all editors "owners." Let me know if you still have trouble.

[walshbr]

Thanks @wcaleb - all done! The bot repo is under PH ownership. Will just need to switch the environment variables to use the PH twitter account whenever we're ready and then update the editorial workflow accordingly to include adding to the spreadsheet.

I'm also happy to add anyone else to the heroku app if the added hands on deck make sense. @mdlincoln maybe? anyone else?

[mdlincoln]

@walshbr You should be able to add my Heroku acct by searching for my email matthew.d.lincoln@gmail.com

[walshbr]

@mdlincoln you should have just gotten an email moments ago - let me know if you don't get a notification from Heroku.

Looks like Heroku charges for resources associated with teams/organizations, while they are free for personal apps. So to keep the thing free we would just need to have a personal app with multiple collaborators. Let me know if anyone else wants to be added.

[jerielizabeth]

@walshbr I have a heroku account (under jeri.elizabeth@gmail.com) so if you need another backup collaborator, you can add me.

[walshbr]

@jerielizabeth - done! Let me know if it didn't go through. The particulars of what is happening are in the README of the proghistbot repo, but I can expand a bit as I'm finalizing.

[acrymble]

@walshbr this seems to be working really well. But I think I might be getting too much of the fanfare given the large number of python lessons. Do you think it's worth adjusting the spreadsheet to give a greater chance for other lessons to appear?

[walshbr]

@acrymble - yeah I was wondering about that as well before we had this start running. One option would be to collapse the Introduction to Python lessons (the original PH sequence) into a single "lesson" row that advertises the whole sequence. That way rather than you having a statistically massive chance of getting tweeted for each piece you'd just get pulled once for the whole thing. That might make more sense anyway, given that those lessons seem to build and depend on each other in a way that few other lessons do. That'd reduce your chances of coming up from 19/62 to 2/45. Does that seem like a workable solution?

That being said, those individual lessons have been getting retweeted a fair amount. So I could go either way, but I think I do agree that it's better to spread the publicity.

[walshbr]

Took care of this @acrymble - not the bot will just refer to that lest set of lessons one time as a chunk.