This Comic Does Not Exist

[zachwhalen]

I've made a comic before, and it's cool to see others are working on them this year as well! This year, I'm trying again, and the rough idea is to use StyleGAN2 to generate some comic panel that I can arrange into pages. Let's see how it goes!

[zachwhalen]

An update. I'm using this implementation for Colab by @dvschultz, and it's really working well. To train it, I downloaded a bunch of EC Horror Comics (Tales from the Crypt, Haunt of Fear, etc.) and ran Fred's multicrop2 ImageMagick script to extract panels.

This worked reasonably well because comics in this set tend to have mostly rectangular panels with little overlap. For example:

All told, I ended up with about 4800 individual panels, but that includes quite a few duplicates due to stories getting printed in multiple issues and editions.

After about 50 hours of training, StyleGAN has gotten pretty good at figuring out panel shape, speech bubbles, and the basics of where text should appear and what it should look like.

Its idea of faces is still a work in progress, but it's gotten pretty good at eyes and mouths. Due to the subject matter of these comics, there are quite a few open mouths and wide eyes, so StyleGAN just needs to figure out how many of each is typical for a page. Here's the current preview:

I don't know how much better I can reasonably expect it to get, but I'll let it run at least for the rest of the day to see what happens. My next step will be to generate panels and arrange them into pages, but I already have some code I should be able to adapt to that purpose.

[zachwhalen]

Interestingly, I also trained it on a set of panels from Watchmen, but the results so far are not as good:

I'm not sure what's happening here, but for some reason it's flattened the color palette into red/orange/yellow and can't seem to get out of that rut. The EC comics dataset went through a similarly weird phrase with washed out color, but it returned to a more appropriate balance eventually. I'm hoping the Watchmen set will do the same if I let it run some more.

[zachwhalen]

I posted a sample latent walk of the current panel generator model on Twitter.

[shawngraham]

When you put together your dataset, how big do you size the images?

[zachwhalen]

@shawngraham The pretrained network model I started with was trained for 1024x1024 images, so I learned (through trial and error) that my images also needed to be 1024x1024. After cropping the panels from the comic pages, I enlarged the images and added padding to make them 1024x1024.

[zachwhalen]

Progress update: I can now generate rows of panels with the ideal proportions. That is, after looking at the layout of some EC pages, it seems that they're mostly making pages composed of 3 rows of 2 or 3 panels each. The opening page and the conclusion for each story is different, but the interior pages aren't usually varying much from that formula. There are interesting exceptions, of course, but for the sake of getting this done, I'm just going to focus on the rows of panels approach.

Now, in my cursory review, it does seem like two-panel rows are more common, but my AI is tending more often to generate panels of the size that better fit into a three-panel row. I'm not sure why, but it probably has to do with balance within the dataset.

Anyway, here's a sample row of three panels:

[shawngraham]

This is amazing stuff Zach.

[zachwhalen]

Well it's "technically done" in the sense that I have 50+ generated comic book pages. Here are a couple of samples:

Sample 1

Sample 2

But I won't call it officially done until I can

1. arrange those generated pages into something like a comic book format, cover included
2. clean up and share my code

[zachwhalen]

It's done! I've got a 55-page comic, which should be plenty if we think of a panel as an image and "an image is worth 1,000 words." It's about 27mb, so Github won't let me attach it to this issue.

For the cover, I trained StyleGAN on a much smaller set. The result yields a partly legible title, but it basically looks like a regular comic cover that someone left out in the rain. Based on the letters I can make out, I invite you to enter ...

VAUDn oc HORRRR

It reads better if you view it two pages at a time in your PDF viewer. I'll post code later once I can clean it up a bit.

Enjoy! IF YOU DARE!

[enkiv2]

This is beautiful. Have you considered having it printed? I would buy it just to have it around my house in case somebody stumbled upon it stoned.

[hornc]

Well done, this is great :) I love the neat page numbering, and the blank second page. Skimming through it and letting the patterns of the words, and their occasionally indeterminate letters, and the panels wash over is quite satisfying. There is this creepy sense of the brown (sometime bluey-green) sharp cross-hatch pattern that is trying to break through and entwine the blobby organic characters throughout the whole work. I think it gets them in the end: "Tob Lat! You're sou soobger...." "You HTAR the EATY pethe..." "... he GOD THEY WON'T wim NAPR!" ... a tragic end

[zachwhalen]

This is beautiful. Have you considered having it printed? I would buy it just to have it around my house in case somebody stumbled upon it stoned.

@enkiv2 I'm thinking about it! There are a couple of features I'd like to add like individual story title pages and story end pages. A few other tweaks, maybe. I've published a book with Lulu before, so I may look into that -- unless there are other options I should be aware of.

And for the sake of completeness, here's some code!

[flarn2006]

I'll just leave this here: