Something Something Cyberwitches

[pjfpotter]

It's definitely going to have cyberwitches. There may be Markov chains. There will be javascript. I may even write some of the code myself.

[pjfpotter]

Spent a very pleasant halloween with the fat moon outside and my children abed, listening to Dan Shiffman teach me of the ways of Markov chains. https://editor.p5js.org/pjfpotter/sketches/RGs2pdK8r

I know have a surprisingly simple markov generator. Enough to generate some quite lovely snatches of prose from a handful of chapters of Dracula.

Do you know where the,moonbeams, were again, and the head as she pointed; the things of,my mouth, and,,jumping to be,away since I closed the ruby

[reakain]

Cyberwitches! Yes please! I can't wait to see where you go with this!

[pjfpotter]

Actually wrote an outline and about ten thousand words for a cyberwitch Academy novel this year, but life beat the bejaysus out of me and it got shelved.

Perhaps it is better this way, a brave and doomed mission into uncharted territory, my cyberwitch cult will emerge...

[pjfpotter]

Thinking about story grid and the fifteen obligatory scenes. Maybe generate a whole load of scene outlines using Kate Compton's tracery.

https://storygrid.com/15-most-important-scenes/

https://github.com/galaxykate/tracery

[pjfpotter]

Must acknowledge Sadie Plant and her book Zeros and Ones for inspiring the idea of cyberwitches.

https://books.google.co.uk/books/about/Zeros_+_Ones.html?id=AEi0AAAAIAAJ&redir_esc=y

[pjfpotter]

So I was somewhat surprised to find that the p5.js web editor could not handle full length novels as input for my markov generator. Is there any way I can fix this? Can I use javascript locally and use my CPU to do the heavy work? How do browsers even work anyway? If anyone is listening, please help. Many thanks.

[cpressey]

So I was somewhat surprised to find that the p5.js web editor could not handle full length novels as input for my markov generator. Is there any way I can fix this? Can I use javascript locally and use my CPU to do the heavy work?

With a big, flashing disclaimer that I've never actually used p5.js or Processing, my first thought is this:

p5.js is an implementation of the Processing programming language in JavaScript, with the intent of running in a web browser. But there are other implementations of Processing too. The original implementation was written in Java, and does not need to run in a web browser. You can download it from the Processing website and try it to see if it works for you.

It might or it might not. The IDE included with the Java version might also have a hard time handling very large text input. Or, there may be divergence between the two implementations such that your code runs on one of them, but not the other. (The p5.js website says only "p5.js is an interpretation of Processing for today’s web" which doesn't suggest they took great pains to remain highly compatible. But who knows.)

[cpressey]

p5.js is an implementation of the Processing programming language in JavaScript, with the intent of running in a web browser.

OK, that's not exactly right. This does exist, but it's called "Processing.js". p5.js is a bit further afield. The idea of p5.js seems to be to use Processing's interface for doing graphics etc, but you write your code in JavaScript directly.

In that case, another implementation of Processing won't help (unless you're willing to translate your code to Processing.)

If you want to run JavaScript code outside a web browser, and you don't really need anything p5.js gives you, then you can run it in node.js.

But if you do want to use what p5.js gives you, then, getting p5.js running under node.js might be possible but sounds likely to be a big hassle, since p5.js probably assumes it's running in a web browser, and all that DOM stuff is missing when you run JavaScript under node.js. There are ways to simulate it but... it's just likely to be a big hassle.

So much for technical solutions! Maybe you can just feed it the input novel in small chunks...?

[cpressey]

Or...! Since it's p5.js's web editor that seems to be the problem, instead of taking the input from it, just take it from somewhere else...?

Like, if the input text is hosted somewhere, like on Project Gutenberg, you could fetch it with an XMLHttpRequest (aka an AJAX call). If p5.js is just JavaScript with some bells and whistles, it ought to allow you to do purely-JavaScript things like this.

Sorry for all the noise. (I learned a bit about the current state of Processing though.)

[pjfpotter]

I'm blown away by the generosity of your answer. It's helpful to hear you think it through out loud so to speak.

My code is only a dozen lines long so I probably could translate it into processing but then maybe will encounter the same limitations. However, my friend codes very large high res images with processing so perhaps it will be able to handle big text files.

I suspect it is possible to use p5.js with node.js but not sure where to start.

Other option is to learn python. But Jesus. I don't think I'll manage that by the end of November lol.

Processing has the advantage of being semi familiar and it has the RiTa library

[reakain]

I'm using python with TextBlob for a lot of my text handling, and there's a nice package called markovify that I've used before too.

Also, more on a conceptual level: if you're doing markov chains and based on the witchy cult comment... Have you considered using markov to build tarot readings, and trying to tell a story through generative tarot? Markov chains work real well to be, like, understandable for lil couple sentence chunks, so maybe use markov to build tarot readings, and then off the first two start building follow on readings that are choosing your markov tarot to match the themeing/message/key words of earlier tarot?

[owenroberts]

@pjfpotter i'm looking at your example and i'm not sure the issue is p5.

i think to start, you need to load the text before you can do anything else in preload, so you shouldn't try to push the result of loadStrings("tracery.txt") to mytext while you're still in preload, you should just use loadStrings and save it to another variable and then do the push in setup. loadStrings and functions like that uses promises to load external files which need to resolve before you can do anything with the results.

after that, i'm kind of confused by your approach. it looks like tracery.txt is actually a JavaScript file? should it just be attached as a JavaScript file in index.html? this looks like you're trying to append the text of the file to the end of the novel. is that the goal?

another thing to note is that the result of mytext = loadStrings("chunkofdrac.txt"); will be an array of string, but the code seems to treat it like a single string. i think you need a nested loop, like in the shiffman example here: https://github.com/CodingTrain/website/blob/main/CodingChallenges/CC_042.2_markov-chain-names/P5/sketch.js

also, i'm not sure but i assume you want to break this text up into words rather than characters, so you may need to create ngrams of word phrases rather than characters.

[pjfpotter]

I've been really ill but I'm back in the room. Thanks so much for your help people.

I found this: https://github.com/shiffman/A2Z-F16/tree/gh-pages/week7-markov/05_markov_mixer

And Shiffman here has solved the problem of clogging up the browser with massive text files by feeding the source texts in word by word, somehow, which involves 5000 divs. Anyway, I'm going to study this.

@owenroberts yeah I think P5,JS is quite a bit more robust than I feared and I now have it comfortably generating markov chains from a novel length input, now using RiTa too. Can now generate over 10k words without crashing the browser. So in theory I can run it five times and get a completed stamp.

@reakain I love the concept idea, I'm just so pressed for time I may have to settle for just stirring up a bunch of different source texts in a big markovian soup

[pjfpotter]

https://editor.p5js.org/pjfpotter/sketches/L1tc8xtwr

[pjfpotter]

AAAAAAHHHH!!!! I FINISHED IT!!!!!!

https://github.com/pjfpotter/CyberWitchAcademy

In the end all I did was feed in 20,000 words of my unfinished cyber witch novel and use RiTa to spit out an enormous Markov chain in Node.

I wish I had had more time to follow all the crazy and fascinating projects of the other entrants. I'll be back next year inshallah.

[hugovk]

CONGRATULATIONS!!

Did you write any code to make it? If so, remember to share it. If not and running existing code, that's fine too, but maybe add links to it?

[pjfpotter]

Yes, sorry I completely forgot about sharing the code. I will upload it tomorrow, even though it is a hot mess. thanks!

[hugovk]

Great, no-one is judging code in any way, and any code that gets the job done is code that gets the job done!

[pjfpotter]

https://github.com/pjfpotter/CyberWitchAcademy

Please find my completed novel and the source code in the repo above.

Thanks @hugovk for organising this fabulous event!

[hugovk]

You're welcome, have a completed label!