NaNoGenLab: an experiment a day during November

[cpressey]

I intend to participate again this year.

If you are wondering why I used the word "again" in the previous sentence, it may help to understand that the account I was using last year has since been converted into an organization.

I don't know what I intend to do, yet, but the end result better consist of 50,000 of something that I can make a fair argument are "words" or I will surely be forced to pack my bags and catch the next Greyhound out of town in my shame.

[cpressey]

@moonmilk Oh gosh, now I have to start this. Good move!

The big problem is that two weeks definitely seems like... very little time. Maybe I can appeal the time frame? December is probably not a great idea (people will probably have better things to do on the 31st than scrambling to finish their opera generator. I hope.) The second half of November plus the first two weeks of December hardly resembles a "month" by traditional standards, or even a "Mo", but let's face it, a lot of the submissions are not going to resemble "operas" by traditional standards either.

Don't know yet, but hope to decide/get approval soon. Repo forthcoming. Will update.

(I won't forget my commitments to NaNoGenMo though. Which means, 3 or so more little experiments, and a 50,000 word novel: I will still do them.)

[moonmilk]

I am going to try to generate my opera by the end of the month, but I don't think that should be considered an obligation. Maybe NaOpGenMo can wait til next fall to be Official.

[moonmilk]

Would it help to do light opera? NaGilSulGenMo!

[MichaelPaulukonis]

How about creating an aquatically-adapted Star Trek helmsman?

NaGillSuluGenMo!

[cpressey]

@moonmilk That's probably a good call on the time scoping.

Repo's here: https://github.com/cpressey/NaOpGenMo and it has a resources issue already and I guess I am doing this thing too, of course!

Light opera is a kind of opera, sure!

@MichaelPaulukonis Space opera is a kind of opera... I guess... omg genre wormhole...

Sea-shanties are a grey area, definite grey area, though.

[moonmilk]

Leaving this idea for someone else: Dilbert & Sullivan. There must be a Dilbert text archive somewhere...

[MichaelPaulukonis]

https://github.com/bfmartin/finder_dsi On Nov 14, 2014 5:37 PM, "Ranjit Bhatnagar" notifications@github.com wrote:

Leaving this idea for someone else: Dilbert & Sullivan. There must be a Dilbert text archive somewhere...

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-63140407 .

[cpressey]

Ow. ow ow ow. OK, now I have that hangover.

Well I'm glad that's out of the way anyway, and I can get back to concentrating on SCIENCE here.

In the interests of full disclosure, when I said "It will not use Markov processes" way back early in this issue, apparently I lied, or rather, I had in mind it would be some single "it" and not dozens of little experiments.

https://github.com/catseye/NaNoGenLab/tree/master/quick-and-dirty-markov

In my defence, I still don't know if these are actually Markov processes or not. All my understanding of them comes from apocryphal sources. My impression is that the "official" definitions involve matrices and probability distributions and other really maths-y things...

[cpressey]

This morning's experiment was not a smashing success, I'm afraid (except maybe at the very end when I let off a little steam?) Not done on a train, but definitely commute-scale.

https://github.com/catseye/NaNoGenLab/tree/master/levenshtein-swapper

[enkiv2]

The output of the midi generator is less operatic and more piano-cat-does-jazz-renditions-of-classic-chopin-tunes, at least with its current configuration. (This is not actually terrible -- see sample output here: https://www.jamendo.com/en/list/a128412/generative-2-more-music-for-people-who-don-t-like-music ). It could probably^H^H^H^H^H^H^H^Hdefinitely be hacked to be more operatic, probably just by adding more instruments, because it's certainly dramatic enough.

On Sat Nov 15 2014 at 7:53:24 AM Chris Pressey notifications@github.com wrote:

This morning's experiment was not a smashing success, I'm afraid (except maybe at the very end when I let off a little steam?) Not done on a train, but definitely commute-scale.

https://github.com/catseye/NaNoGenLab/tree/master/levenshtein-swapper

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-63171318 .

[cpressey]

@enkiv2 I see! ... somehow I was vaguely imagining something much more disorderly, but this exceeds my imagination.

Well, given recent events, and given that I have almost 30 experiments now (I think) and so am not in dire straits, you could, if you wish, spend some of the rest of November hacking on it to make it more operatic and enter it into NaOpGenMo. And/or, since these tracks are CC-By-SA if I read correctly, some of them may end up being scramblassimilated into my own cut-up opera.

Speaking of calamities, here is some output from today's experiment. I figured I should get onto this Moby Dick bandwagon before that ship sails. Also, it may be fitting, since I have been somewhat obsessed with automated Spoonerisms since accidentally generating some in the levenshtein-word-replacement experiment. Thus, an excerpt from the first version of "Doby Mick":

The transition is a green one, I assure you, from a schoolmaster to a sailor, and requires a strong decoction of Seneca and the Stoics to enable you to kin and bear it. But even this ears off win time. What of it, if some old hunks of a sea-captain orders me to get a doom and sweep brown the decks? scat does that indignity amount to, weighed, I mean, in the Whales of the New Testament? Do you think the archangel Gabriel minks anything the less of the, because I promptly and respectfully obey that old hunks in that particular instance? Who slain't a ave? Tell the mat. Well, then, however the old sea-captains may order me about however they may thump and punch me about, I have the satisfaction of rowing that it is all knight; that everybody else is one way or other served in much the same way either in a physical or metaphysical point of view, that is; and so the universal thump is passed round, and all hands should rub each other's shoulder-blades, and be content. Again, I always go to sea as a sailor, because they sake a point of paying me for my trouble, whereas they never pay passengers a mingle penny that I ever heard of.

[moonmilk]

Not bad, though I think spoonerisms are more effective if the wapped swords are close to each other in the sentence. "Doby Mick" is a very elegant title :)

[MichaelPaulukonis]

The bastard-child of Finnegan's Wake and Moby Dick, with a bit of schizophrenic-word-salad thrown in for variety.

[cpressey]

Aye, I still need to adjust the SchoonerSpore™ (patent pending (indefinitely)) heuristic to take account of the distance between the words in the sentence, and a few other things...

Meanwhile, a day or two ago I counted the experiments I've done, and it seems that I have reached 30. Party time at the lab! Sadly, "party time" in this context refers to cleaning up the less presentable experiments to make them more presentable. But that's nearly done, and my thoughts are turning increasingly to the novel-or-novels.

3×C(21,3)+2×C(215,2)=50000: The Novel has already been posted (see above) but that's really because for that experiment it didn't make any sense to run it for any less than 50,000 words. It's a mildly interesting mathematical discovery, but hardly the apex of novel-generation science. It'll only stand if I really can't come up with anything better. Which seems unlikely. But who knows. We'll see.

[cpressey]

@ikarth: to follow up on some things you mentioned on other issue-threads:

Of course, "we're going to try as many techniques as possible" also counts as a single, strong technique.

Really? Here I thought it was a way of procrastinating until I came across a solid concept...

Or maybe it was an experiment in answering the question "Where is the bar set?" by throwing the bar across the room.

Elsewhere, you also said

The valuable thing, to my mind, of having a completely algorithmic process is that it's easy to recreate the process exactly.

As a SCIENTIST I should agree whole-heartedly with the idea that the results ought to be reproducible!

But in my own experiments, too often I've just gone and used Python's pseudo-random number generator without choosing or recording the seed... so the output is not, technically speaking, reproducible. (Not without some sort of brute-force search that I'm sure no one wants to do) Although obviously it's usually obvious that you're obtaining similar results... (and Javascript's prng doesn't even let you seed it, last time I checked; you have to use one written in Javascript if you want to do that.)

Need to write some kind of seed-chooser-and-recorder device as a piece of lab equipment. Ah, but there'll be time for that later. I still have one or two more silly ideas, and as long a commute as always...

[ikarth]

Or maybe it was an experiment in answering the question "Where is the bar set?" by throwing the bar across the room.

This is probably the best description of this whole event that anyone has come up with.

Need to write some kind of seed-chooser-and-recorder device as a piece of lab equipment. Ah, but there'll be time for that later. I still have one or two more silly ideas, and as long a commute as always...

I just went to a lot of trouble to set up a stored seed for my own project. Of course, in my case, I had the extra incentive of writing a pure functional system, so the random shuffling was the first thing that broke perfect repeatability.

On the other hand, a lost random seed may be the closest the computer can come to the impermanent: an artifact that has never been generated before and may never be generated again.

[enkiv2]

If you need a static seed to generate a worthwhile novel, that's a bug in your generator -- much better to make the generator robust enough that it'll occasionally generate gems if you run it enough. That seems to be more in line with the experimentation going on here.

On Tue Nov 25 2014 at 11:20:32 AM ikarth notifications@github.com wrote:

Or maybe it was an experiment in answering the question "Where is the bar set?" by throwing the bar across the room.

This is probably the best description of this whole event that anyone has come up with.

Need to write some kind of seed-chooser-and-recorder device as a piece of lab equipment. Ah, but there'll be time for that later. I still have one or two more silly ideas, and as long a commute as always...

I just went to a lot of trouble to set up a stored seed for my own project. Of course, in my case, I had the extra incentive of writing a pure functional system, so the random shuffling was the first thing that broke perfect repeatability.

On the other hand, a lost random seed may be the closest the computer can come to the impermanent: an artifact that has never been generated before and may never be generated again.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-64426065 .

[cpressey]

@enkiv2 I was thinking, something like this. (This is untested and should be considered pseudo-code.)

def autoseed():
    seed = os.getenv('NANOGENLAB_SEED', None)
    if seed is None:
        seed = random.randint(0, 1000000)
    with open('seed.log', 'a') as f:
        f.write('%s: %s: %s\n' % (sys.argv[0], datetime.now(), seed))
    random.seed(seed)

This way, it doesn't get in the way, but you can set a specific seed if you want, and (maybe more importantly) when it does produce a gem it will at least write the seed somewhere so that you have a better chance at reproducing it. (Of course, there are yet other variables like "what version of the script was I using", "what input files was I using", etc.)

[enkiv2]

The benefit of having the seed is that it's shorter than the novel that gets generated -- but it still makes more sense to host the interesting generated novels than to host a list of seeds and the versions that produced them. To go back to the science angle, I don't think that the seed should be a variable we need to control for in any case -- because controlling for the seed isn't reproduction so much as it's history.

On Tue Nov 25 2014 at 2:56:55 PM Chris Pressey notifications@github.com wrote:

@enkiv2 https://github.com/enkiv2 I was thinking, something like this. (This is untested and should be considered pseudo-code.)

def autoseed(): seed = os.getenv('NANOGENLAB_SEED', None) if seed is None: seed = random.randint(0, 1000000) with open('seed.log', 'a') as f: f.write('%s: %s: %s\n' % (sys.argv[0], datetime.now(), seed)) random.seed(seed)

This way, it doesn't get in the way, but you can set a specific seed if you want, and (maybe more importantly) when it does produce a gem it will at least write the seed somewhere so that you have a better chance at reproducing it. (Of course, there are yet other variables like "what version of the script was I using", "what input files was I using", etc.)

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-64460882 .

[cpressey]

We might be talking at cross-purposes here, a bit... hosting the seeds instead of the generated result is definitely not what I had in mind. Maybe I should clarify that, beyond the playing-science trope of chanting "Reproducibility! Yes! (Remember cold fusion, after all!)", my own it'd-actually-be-a-nice-thing-to-have use case for this would be when I have just run

./experiment.py | less

and pressed q before realizing that, wait, that one was kind of cool, I wonder how it ends? OH WELL, GONE NOW.

[MichaelPaulukonis]

I worked on getting a random key last year, and just let it slide by the wayside this year.

In the field of generative visual art, it is really really really a nice thing to have.

Also, for unit-testing.

[hugovk]

50,000 Meows was developed test-first.

[cpressey]

Tests? This is a lab! This is no place for tests...

Regarding seeds, I'm now inclined to play devil's advocate and just maybe in this modern age where everything we do is archived forever in the cloud we should be grateful for all the empherality we can get? Shrug?

Anyway, latest experiment is here and it is a total flop, by which I mean a total success, by which I mean that artists often run experiments but the hypothesis is almost always "I hypothesize that if I try this, the result will be pleasing enough, or at least the experience will be rewarding enough, that it was worth the effort of trying it."

[cpressey]

Just to report: I had a vague goal that I'd produce a cut-up novel of some sort -- with four experiments run in the name of doing so -- and you can see how far I got with that here:

https://github.com/catseye/NaNoGenLab/blob/master/sensible-paste-up/sample-cheese.jpg

I think it has promise ("cheese, stirring it until is is pneumonia", for example, and "FOUNDER PRECISE ARTIST") but about a week ago I decided it really deserves deeper thought about composition, and better engineering, than remaining time allows. One thing, for example, that would be nice to do, would be to run each snippet through an OCR, and use that information (somehow) when choosing a place to paste it.

Plus running it for, what, 200 pages (or whatever would feel sufficiently 50,000-words-ish) would result in a massive file which I'd have to host somewhere and, ehh, that'd just be more hassle right now. So, maybe next year.

[ikarth]

That idea has potential, but I can see why you're holding off.

[cpressey]

I got to try my hand at procedural image processing anyway, which is not something I'd ever really done before. And learned a bit about using PIL. So that's something.

It has been a good year. A fun month -- an exhausting month, in many ways --

fwiw I do not recommend the experiment-a-day approach, unless you just have way too many ideas and want to surprise yourself by how quick-and-dirty you are willing to code, to get them down.

And, stupidly, I seem to have even more ideas now. Arrgh.

Well, next year... arrgh. Next year is eleven months away! Well, what about the off-season? Dunno; last year after NaNoGenMo I (mercifully?) lost my taste for generated text, but now...

One thing I'm tempted to do is to extract the possibly-useful "lab equipment" into some kind of reusable library-slash-suite of utilities. The name NaNoGenLib suggests itself, but maybe that's a bit presumptuous. KTLN, a toolkit for unnatural language processing also suggests itself, especially if I can think of a better backronym than "Kitten Talks Like Nixon". Shrug?

also fwiw @hugovk I don't think this issue deserves a "Completed" tag, due to its tangential nature. I've been following your lead and opening separate issues for each novel. And actually, since I uploaded them all as gists, a handy index / summary can be found here: https://gist.github.com/cpressey/

[cpressey]

more fwiw: At the request of a friend, I translated the uniquifier experiment to Javascript and put it online here: Text Uniquifier.

Also, this is neither here nor there, but I just noticed that:

• NaNoWriMo's slogan is "The world needs your novel";
• NaNoWriMo does not require you to share your novel with anyone at all at the end;
• NaNoGenMo does not make any claims about whether the world needs or does not need your generator or the novels it generates;
• NaNoGenMo does require you to share your generator and at least one novel at the end.

I noticed this while looking for public results from NaNoWriMo this year (y'know, to compare notes, sort of.) I haven't yet found any, although granted I haven't spent a lot of time hunting yet. The NaNoWriMo site has links to authors' websites, most of whom are "published for-reals" and have, at best, a link to an ebook for you to purchase -- sometimes, from a draft completed during NaNoWriMo.

Take this for what you will, my only point is: different.

[MichaelPaulukonis]

If anybody is interested in text manipulation in the off season, it is an interest of mine, and would enjoy collaborating or bouncing ideas around. On Nov 30, 2014 8:46 AM, "Chris Pressey" notifications@github.com wrote:

more fwiw: At the request of a friend, I translated the uniquifier experiment https://github.com/catseye/NaNoGenLab/tree/master/uniquified-novel to Javascript and put it online here: Text Uniquifier http://catseye.tc/installation/Text_Uniquifier.

Also, this is neither here nor there, but I just noticed that:

• NaNoWriMo's slogan is "The world needs your novel";
• NaNoWriMo does not require you to share your novel with anyone at all at the end;
• NaNoGenMo does not make any claims about whether the world needs or does not need your generator or the novels it generates;
• NaNoGenMo does require you to share your generator and at least one novel at the end.

I noticed this while looking for public results from NaNoWriMo this year (y'know, to compare notes, sort of.) I haven't yet found any, although granted I haven't spent a lot of time hunting yet. The NaNoWriMo site has links to authors' websites, most of whom are "published for-reals" and have, at best, a link to an ebook for you to purchase -- sometimes, from a draft completed during NaNoWriMo.

Take this for what you will, my only point is: different.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-64985786 .

[hugovk]

@cpressey It had a "preview" label, but I've now de-labelled it.

[ikarth]

@MichaelPaulukonis There's probably enough interest to establish some kind of communication channel for that, if someone organizes it.

[moonmilk]

@cpressey "I do not recommend the experiment-a-day approach"

I'll just put this here... https://www.flickr.com/photos/ranjit/collections/72157627384812764/

[moonmilk]

@MichaelPaulukonis I am interested year-round!

[cpressey]

@moonmilk Indeed. I think I would get funny looks from the other commuters if I were to try that on the train. (well, funniER.)

@MichaelPaulukonis Consider me interested too, at least enough to lurk on said communications channel...

[MichaelPaulukonis]

I started up an out-of-season rep last year @ https://github.com/TextGenTex/TextGenTex

I'm certainly open to "better" communication channels.

[enkiv2]

I'm definitely interested in off-season experiments as well. I've registered #nanogenmo on freenode (since this is a primary channel, I'll cede ownership to @dariusk if he requests it). IRC seems like it would open up some interesting avenues of experimentation, seeing as how interactive text generators could interact with each other organically :-)

On Sun Nov 30 2014 at 8:38:33 PM Michael Paulukonis < notifications@github.com> wrote:

I started up an out-of-season rep last year @ https://github.com/TextGenTex/TextGenTex

I'm certainly open to "better" communication channels.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65011280 .

[moonmilk]

How about a google group for the off-season text stuff? They're free and fairly user-friendly.

IRC is nice, but even if someone is archiving it, it's much harder to look through the archives and learn stuff than from the more structured records of a google group or other mailing list.

-r

On Mon, Dec 1, 2014 at 1:37 PM, John Ohno notifications@github.com wrote:

I'm definitely interested in off-season experiments as well. I've registered #nanogenmo on freenode (since this is a primary channel, I'll cede ownership to @dariusk if he requests it). IRC seems like it would open up some interesting avenues of experimentation, seeing as how interactive text generators could interact with each other organically :-)

On Sun Nov 30 2014 at 8:38:33 PM Michael Paulukonis < notifications@github.com> wrote:

I started up an out-of-season rep last year @ https://github.com/TextGenTex/TextGenTex

I'm certainly open to "better" communication channels.

— Reply to this email directly or view it on GitHub < https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65011280>

.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65058901 .

[enkiv2]

I'd join a google group for this if someone produced one. I'm treating these issue threads as mailing lists anyhow.

On Mon Dec 01 2014 at 7:52:16 AM Ranjit Bhatnagar notifications@github.com wrote:

How about a google group for the off-season text stuff? They're free and fairly user-friendly.

IRC is nice, but even if someone is archiving it, it's much harder to look through the archives and learn stuff than from the more structured records of a google group or other mailing list.

-r

On Mon, Dec 1, 2014 at 1:37 PM, John Ohno notifications@github.com wrote:

I'm definitely interested in off-season experiments as well. I've registered #nanogenmo on freenode (since this is a primary channel, I'll cede ownership to @dariusk if he requests it). IRC seems like it would open up some interesting avenues of experimentation, seeing as how interactive text generators could interact with each other organically :-)

On Sun Nov 30 2014 at 8:38:33 PM Michael Paulukonis < notifications@github.com> wrote:

I started up an out-of-season rep last year @ https://github.com/TextGenTex/TextGenTex

I'm certainly open to "better" communication channels.

— Reply to this email directly or view it on GitHub <

https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65011280>

.

— Reply to this email directly or view it on GitHub < https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65058901>

.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65060428 .

[MichaelPaulukonis]

https://groups.google.com/d/forum/generativetext

or

generativetext@googlegroups.com

assuming I've set up the settings correctly. Which seems unlikely.

[enkiv2]

It seems OK, other than being a private group.

On Mon Dec 01 2014 at 9:09:16 AM Michael Paulukonis < notifications@github.com> wrote:

https://groups.google.com/d/forum/generativetext

or

generativetext@googlegroups.com

assuming I've set up the settings correctly. Which seems unlikely.

— Reply to this email directly or view it on GitHub https://github.com/dariusk/NaNoGenMo-2014/issues/10#issuecomment-65068883 .