Constrained writing

[cpressey]

Consider an exercise in constrained writing defined by the following axiom system N:

C1. The final written product contains at least 50,000 words.
C2. The final written product is believed by its author to be a novel.
C3. The writing of the product takes place during the calendar month of November.

(Players of the Glass Bead Game will immediately recognize this as a residulated finite premagma over 𝕌ₔ, but let's not get ahead of ourselves.)

Note that we can form a conservative extension N′ of N by adding an additional axiom:

C4. The writing of the product must be done by computer code acting in the capacity of ghostwriter.

Note also that N′ can be characterized by its basis of fudges:

• What even is a word? (cf. "meow")
• What even is 50,000 words? (cf. "junk meows")
• What even is November? (cf. "cheating non-entry")
• What even is a novel? (cf. "how do I generate PDF?")
• What even is computer code? (cf. "language survey")

Now let us consider a non-standard model of N′ where any string of words may be considered a novel. This is equivalent (up to isomorphism) to restating C2 as:

C2′. The final written product is believed by its author to be a string of words.

At first blush C2′ appears to be a tautology. But is it really?

Yes. Yes it is. We can remove it entirely. But then, note that our finite premagma is no longer residulated! Is it not a famous open problem whether a non-residulated finite premagma over 𝕌ₔ contains any non-trivial members at all?

Thus we posit the existence of a higher-order constraint:

C0. Bring your own C2′.

This suggests the existence of several other higher-order constraints:

C(⁻1). Think up a C2′ that is interesting enough that you would want to see it through to the end, but simple enough that you could finish it in your spare time in 30 days. C(⁻2). Have a greater-than-zero amount of spare time in the 30 days constituting the calendar month of November.

It perhaps goes without saying that if C(⁻2) is not true, none of the other axioms in N or N′ can be fulfilled.

But if it does go without saying, then why am I saying it?

[cameronedmond]

As always, I'm struggling to comprehend your project. But, as always, this means I'm very excited by it.

[cpressey]

As always, I'm struggling to comprehend your project. But, as always, this means I'm very excited by it.

Trust me, I'm as confused as you are. I mean, right now I don't even know if my novel is going to be generated in the EU or not. How messed up is that.

[sbassoon]

C0. Bring your own C2′.

YES i love this! looking forward to more

[cpressey]

YES i love this! looking forward to more

Okay! I conjecture that we can achieve an even tighter upper bound for non-trivial residules over N′ if we stipulate the following additional higher-order constraint:

C(⁻3). In many parts of the world, November is the start of cold-and-flu season.

[cpressey]

Trust me, I'm as confused as you are. I mean, right now I don't even know if my novel is going to be generated in the EU or not. How messed up is that.

Well, November's here and it looks like the "Novel is to be generated in the EU" constraint is, in fact, on.

Assuming I have any hopes of being the first Canadian to generate a novel for NaNoGenMo in the UK but not in the EU, I suppose they'll need to wait until next year.

This concludes the requisite conceptual dealings with the "Na" part of "NaNoGenMo".

[cpressey]

Humpday on the first full week. Traditionally, this is the day we stand in a circle 'round the repository, hold hands and sing D'skuban methane-shanties. Then we tell each other reasons that we think we're not vampires ("Darius, I think you're not a vampire because.... you don't have sharp teeth!" "Hugo, I think you're not a vampire because... you don't like the taste of blood!" &c.) Finally, we cap off the evening by setting the "Resources" issue on fire and roasting marshmallows over it.

However, this year we may as well dispense with tradition and get straight on with PRODUCING MASS QUANTITIES.

For my part, I've gotten as far as creating a repository and I hope to have a generator and a novel in it by the end of this the first week, knock on wood.

[cpressey]

For my part, I've gotten as far as creating a repository and I hope to have a generator and a novel in it by the end of this the first week, knock on wood.

Nope, didn't happen.

[cpressey]

I believe I finally have an idea. I mean, one that is both doable in what's left of November, and one that I want to do. It's somewhat technical so I don't expect a spectacular novel to come out of this, but I'm OK with that.

Some background:

Markov processes are a popular technique in generative text. It's easy to obtain one from an input corpus, but famously they can't capture simple grammatical rules such as that every closing parenthesis should be preceded by an opening parenthesis.

Formal grammars are also a popular technique, and unlike Markov processes they can describe and enforce grammatical rules. But obtaining a grammar is not as simple as reading through a corpus and recording word occurrences.

My idea here is to formulate a kind of compromise.

Say we are given a regular grammar which defines some minimal grammatical rules, such as for open and closing parentheses, and open and closing quotes. This can be thought of as a state machine which is, at any given time, in one of a number of states.

Then, the idea is to give each of these states its own dedicated Markov process, extended in the following modest way: it associates to each word, not just a weighted set of words that could come next, but also whether any of these words shall cause the state machine to transition to a new state.

This thing is surely not novel, so it might well have an established name... but if so, I don't know what it is. You can't really call it a "Markov" anything anymore, since it's no longer "memoryless". You could probably call it a finite-state stochastic process though.

Again, I'm settling on this idea not because I expect that it will produce stellar output -- it will look like typical Markov-process output, except with fewer punctuation errors, I imagine -- but because it's both moderately interesting to me, and doable.

[cpressey]

Again, I'm settling on this idea not because I expect that it will produce stellar output -- it will look like typical Markov-process output, except with fewer punctuation errors, I imagine

Initial experiments suggest it's a little better than that. In a simple state machine based on quotation marks (see diagram below), there are essentially two Markov chains, one for narration and one for dialogue. Since those have different characteristics, when generating from the narration model it looks a bit more like narration, and when from the dialogue model it looks a bit more like dialogue.

[cpressey]

Half! Time! Update!

I had planned to post some substantial progress report on this day, but work got busy, so it'll have to be brief.

Basically, it works. By default it uses the "narration/dialogue" state machine depicted in the previous comment, but telling it to use a trivial state machine instead makes it equivalent to a Markov chain (of order 1).

To illustrate: populating the model with Anne of Green Gables, here is some Markov chain output:

When Anne, we make life returning meekly. Deep down by its warm jacket were to me a chalk crayon and she said Marilla,” said Anne, so pale little orphan asylum? Marilla is no longer.” began her. I suppose my time this Matthew’s unaccountable whim and found and I foresee that?” said. The walls are Moody Spurgeon wandering happily, except Carrie Sloane.”

The Fairy Queen” said Marilla, there is such a bad as she knows all over the spare room door. Mrs. Spencer was none,” protested Anne.

As you can see, there's no distinction between narration and dialogue, and quotation marks are not balanced. To compare, here's some output from the finite-state stochastic model.

“I couldn’t settle the times he must go to bring a skunk cabbage.” persisted Anne about it would have fled to withdraw her feet. “A-o-bag. You must win, golden hair a decent color at home. It will forever and she supposes we are the mint delicious? I wouldn’t forgive you done, with such a complete confession—oh, he’s really think of anything for imagination. Mrs. Spencer bring us royally. This is awful sick over at night.”

Anne fled to her room, lifting her own children up proudly to be begun among them. Tonight a prayer of dumbness with a promising friendship existed and Anne gathered some time forgetful of something happened not at her eyes.

[cpressey]

Over the weekend I was thinking about this, and I saw more clearly how it relates to a conventional Markov chain.

If you look at a Markov chain as a probabilistic finite automaton, what you're doing when you construct an order-2 Markov model is, basically, performing the product construction for finite automata. You have states A and B in one automaton, states C and D in another, and your new automaton has states AC, AD, BC, BD.

In the model I've been working with, the finite state machine that represents the regular grammar (quotation marks and such) is also a finite automaton; so you can basically perform the product construction on it and the Markov model, and get a new Markov chain which has all the properties of the regular grammar encoded in it.

For example, you might have a text

Then she said, "I see she isn't here."

You can run the state machine over this to tag each lexeme with the grammar state it's in (N=narration, D=dialogue)

NThen Nshe Nsaid N, N" DI Dsee Dshe Disn't Dhere D. D"

In the first text, the two "she"s are not distinguishable by a Markov chain. But the Nshe and Dshe in the second text are, and when making the model, they'll each get their own probability distributions leading up to and following them.

In short, after tagging, you can treat it like an ordinary Markov chain. Besides being generally simpler, this also means -- for whatever it's worth to anyone who's not a statistician -- that it retains the Markov property. At the very least, we can still call it a Markov chain, and there's something to be said for familiar terminology. All that remains is to untag the lexemes before producing the final output, to make it presentable.

However. This only works because this is a regular grammar. (I wanted to start small). If you wanted to impose the structure of a context-free grammar on a Markov chain, you'd still need some mechanics that go beyond this; you'd still lose the Markov property. However however, it's rare to see conventional texts where quotations and parentheticals are actually nested arbitrarily deeply. The spoken language's phrase structure sure can be, but I have a hard time seeing what kind of CFG you could pick that would apply well in that case.

So, not sure which direction I'll pursue in the next 2 weeks, but happy to see that this model reduces to a simpler thing.

[cpressey]

Hark! Is that the sound of the chancery bell striking twenty o'calendar? Indeed it is, marking...

Two! Thirds! Day!

This is the day that -- having now satisfied ourselves that we are not surrounded by vampires (at least on paper) -- we begin constructing a giant effigy of The Written Word out of whatever wood, sticks, branches, twigs, &c. we have laying around. After it's completed, at the end of the month, we will of course unpack our hockey sticks from our duffel bags, grin like ninnies, and proceed to whack it to bits, hoping beyond hope that from it will burst forth a veritable cateract of goodies, in the manner of, if not a piñata exactly, then perhaps like the end of an exceptionally well-played level of Bubble Bobble.

I cannot say with certainty that we are always disappointed in this.

But again, this year we might as well dispense with tradition and just get on with the ABJECT PANIC.

For is it not around now when many of us seriously consider the following line of action: slinking away and telling ourselves that that green "Completed" label was probably sour anyway? Is it not around now when we look over all the intricate models and beautiful pipelines we've assembled before ourselves, frown in dismay at our estimation that it will take at least 15 more days to get everything in without a clash, and resort to throwing together some gag novels merely "for the lulz" (as the kids say)?

[I]n a world of information overload, what cuts through are stunts. -- Amol Rajan, BBC media editor

[cpressey]

Alas, even the route of throwing together some gag novel generators is more hazardous this year: people'll expect 'em to fit into 256 characters now!

I have nothing against code golf, but I don't have a lot of spare time this year, and that's not what I want to spend it on.

I've been using what spare time I do have to try to solve the problem of how to turn this "finite-state-machined" Markov chain into an order-2 Markov chain. The solution I've got is to use streams of "tagged tokens". First, from the input text, produce a stream of tokens. Then go over this stream and, stepping through the FSM, tag each token with the state it's in. Then go over it again and, remembering the last token on each step, tag each token with the last token. Then build the Markov model from that, tags and all. (Then remove all the tags from the output stream before presentation).

I think that's all that's needed, but there may be a subtlety I've missed. Need to implement it and see. If it doesn't work, will probably abandon it and keep with the order-1 Markov chain.

[cpressey]

Well it's 93⅓% Day today and... yeah yeah, you know the drill by now, so I think we can just skip straight ahead to putting the finishing touches on this axiom system we've been working on here:

C(⁻4). It's a bit hard to code while under the influence of a tryptophan/hyperconsumerism speedball.

And I do believe that completes this little journey into the conceptual underpinnings of NaNoGenMo as an exercise in time management constrained writing. I just wanted to say, you folks have been really great, it's been really great promoting myself here this year, follow me on social media, buy my book, tell your friends, subscribe to my newsletter, my real-time update alerts are really great and the best thing about them is that they're completely 100% vampire-free.