What If: An attempt without any code

[rasagy]

I always wanted to collect statements around "What if I had done blah blah", thinking of the possible timelines that we avoided (or wished were true). So I decided to search through twitter for tweets with a phrase above, and maybe compile it into a book of regrets. :D

Problem: I don't code often (Just simple sketches on Processing, and a basic Javascript), so I'm instead going to take a shot at doing this without writing any code and using easy-to-pickup tools. As of now, I'm looking at IFTTT and Excel (maybe some Sublime Text for quick Find+Replace to clean the data).

fingers crossed

[rasagy]

Update: Created a IFTTT recipe to search tweets & append a json and store it in my dropbox. Got about 150-200 tweets in 12 hours, so this will take a while.

Search query: ("what if I had" OR "What if you had" OR "What if she had" OR "What if he had")

Saved as:

{"tweet": "{{Text}}", "created_at": "{{CreatedAt}}", "user": "{{UserName}}", "user_dp": "{{UserImageUrl}}", "link": "{{LinkToTweet}}"},

Using a json instead of csv because I'm slightly unsure about extra commas in the tweet messing up the import. Realized later that the tweet text can also have "", so restarting the recipe with a fix on that (I'll later replace the " with \"). Also need to deal with SPAM (added a few keywords to the search query right now, but will deal with this later).

Set up a Github repo for the data and progress: https://github.com/rasagy/what-if

[hugovk]

Good idea to use IFTTT, I used it to make my first Twitter bot.

[rasagy]

Stumbled on the Whim project (What If Machine) while reading a Guardian piece (that kind of mocked NaNoGenMo :-/) Interesting take at generating statements like

What if there was a little bicycle who forgot how to move? (Much more interesting than most tweets I'm collecting). * sigh *

I'm still going to go ahead with this, though the tweets are coming in slow (and filled with spam). Oh boy.

[hugovk]

You may have better luck harvesting using twarc, "a command line tool for archiving JSON twitter search results".

It's a Python script, so you need to install Python, and you'll need some API keys from Twitter, but it would be a good way to get loads of source data and doesn't require any programming.

[cpressey]

From that Guardian piece, apparently in reference to NaNoGenMo:

“I don’t think anyone’s really taking it seriously,” says Mark Riedl, an associate professor at the Georgia Institute of Technology.

Easy for him to say -- he's got funding!

Whereas all I've got is a long commute.

(Oh, and a gigantic Lab Complex inside the hollow Earth, but never mind that. I'll show them! With some help from the Deros maybe...)

[enkiv2]

Is it just me, or is there not really a great deal of difference between the projects that article talks up and some of the ones we're producing, in terms of quality?

On Wed Nov 12 2014 at 6:24:03 AM Chris Pressey notifications@github.com wrote:

From that Guardian piece, apparently in reference to NaNoGenMo:

“I don’t think anyone’s really taking it seriously,” says Mark Riedl, an associate professor at the Georgia Institute of Technology.

Easy for him to say -- he's got funding!

Whereas all I've got is a long commute.

(Oh, and a gigantic Lab Complex inside the hollow Earth, but never mind that. I'll show them! With some help from the Deros http://en.wikipedia.org/wiki/Richard_Sharpe_Shaver maybe...)

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/88#issuecomment-62704520 .

[MichaelPaulukonis]

Easy for Reidl to say -- and he never posts any of his code in public!

There's a lot of whimsy in here -- I've commented on that elsewhere. But that's not to say that we aren't taking it seriously. The rigour of the competition is in the speed -- one month for a novel algorithm is FAST AND FURIOUS compared to the years and decades some of the extant systems have been under construction.

And in comparison, the readability of pieces generated herein is pretty d****d good!

[ikarth]

@MichaelPaulukonis It may just be because it's more accessible and has put more projects on my radar, but from my perspective NaNoGenMo has engendered a huge explosion of projects in this space.

[rasagy]

IMO having a platform like this has it's own value: You now have a bunch of folks who have experimented in multiple fields (& not a single domain that such labs venture into) related to this, in the same place. That's great for beginners (like me) to refer/get inspired, as well as experts to bounce of ideas or discuss implementation details. Not taking this seriously is quite frankly, kiddish.

[cpressey]

@rasagy Well put. I'll add that the range of skills/experiences involved is quite broad -- anyone who is an expert in one sub-area is likely also a novice in some other sub-area -- so there's always something new for anyone to follow up on, if they feel so inclined.

@MichaelPaulukonis I have several thoughts in response, but what if :thought_balloon: I try to collect them on my own issue instead of continuing to clutter @rasagy's issue with them. (Github doesn't make the best forum, although it's definitely interesting to try to use it as one.)

[rasagy]

@cpressey Haha, well there is @MichaelPaulukonis's issue around Press Coverage #92 which might be the best place to continue this. Hopefully someone will write a more balanced piece (on this year's edition) that we can all point others to. :)

[rasagy]

Got a little late to finish this. Quick update:

• Got two sets of tweets: One specific to "What if [x] had", and a more open ended "What if"+"had" using IFTTT. Decided to use the latter (more tweets)
• Roughly 34K tweets from Nov.10 to Nov.30 (including RTs)
• Used a bunch of regex to fix things like removing links, escaping " etc. but still ended with errors in json
• Decided to convert everything to csv
• Played around in Excel to remove RTs, extract phrases that start with "What if" and end with a punctuation
• Used random() to assign some tweets special formatting (and some hand picked tweets too) to avoid generating just a huge chunk of text
• Converted to html, added intro section, made minor changes in word (Page numbers etc.) and made a pdf.

phew

[rasagy]

Final novel here: http://rasagy.in/what-if/ PDF version here: https://github.com/rasagy/what-if/blob/gh-pages/What%20if%20-%20NaNoGenMo%20entry%20by%20Rasagy.pdf?raw=true (Trimmed to 133 Pages, ~58K words out of ~170K words)

This was fun & frustrating at the same time. Will blog about pitfalls to avoid if you're going to be stubborn about not using code. :)

[hugovk]

I like it. Using random what-ifs as bold headings and italic subheadings works well to arbitrarily break a wall of text up.

[MichaelPaulukonis]

Will blog about pitfalls to avoid if you're going to be stubborn about not using code.

Did you blog?