Art: WFC/Wave function collapse city

[avaer]

https://github.com/marian42/wavefunctioncollapse

This is a good Unity repo + codebase for procedurally generating cities, better than what I had planned. It essentially uses a set of prefabs to glue together a cityscape based on fills and heuristics.

The actual blocks FBX is here: https://github.com/marian42/wavefunctioncollapse/blob/master/Assets/blocks.fbx

We can use this to fill the need to have a city in Zone 0, meeting the last requirement for the trailer.

The work involved is:

• [ ] create a more solarpunk set of prefab blocks, by editing the existing blocks.fbx; we might also want to create the textures at this stage
• [ ] update the algorithm to increase height a bit
• [ ] generate a bake
• [ ] fix textures (if needed) and and edges to create the final city model

Inspiration

[madjin]

Familiarizing myself with this project, so far set it up and usied https://github.com/Plattar/gltf-exporter to export as GLTF

124k triangles from a 16x16 area generation

[madjin]

More pics:

Could also generate these more vertically as well

[avaer]

Great research, looks sick. 😍 I’m curious how this textures. If you uv/texture the source does it come out in the model?

Once we understand the process, we should redo the ingredients and textures to be strong “Webacity”/“Zone 0”/“Nihon” architecture.

It would take a bit of work, but the resulting effect might be pretty magical.

[madjin]

I did a bit of testing to see what the optimal workflow. When I first attempted to modify the pieces inside Unity, lets just say it didn't like that and I had to nuke / redownload the entire project.

Meshes

The blocks.fbx file is hard to edit in Unity and blender because when you open it up it looks like this:

Gotta open up the prototypes file where all the pieces are individually arranged

You can then change values in the ModulePrototype component where you can see a preview of what it looks like in the context of other pieces. Great way to pre-visualize the generation.

It took me a bit to figure out how to modify the meshes, both blender and unity have different scale / coordinate systems.

First, figure out what piece you want to edit based on step above so you know how it'll fit with other pieces. Then, import the FBX file into blender with Apply Transform box checked.

Search for the piece you want to edit, then hide everything else to make it easier to work on just that. Make sure whatever you're doing is joined to that specific piece.

When done, unhide everything again and export with Apply Transform box enabled again. You can name the fbx file whatever you'd like.

Import custom FBX

When you import the FBX into Unity make sure to uncheck Convert Units and check Read/Write enabled.

Open up the Prototypes.unity project and drag the Prototypes.prefab into the Hierarchy. Select the piece(s) that you modified and replace the Mesh Filter with your new piece from the FBX you imported (mine was blocks7.fbx).

Save the project. You can now open the Game.unity project and generate with the modified pieces.

Texturing

I'm still figuring out the workflow for texturing modular pieces before generation, something seems to break which causes them to disappear in the output. In the meantime after generating you can either texture in Unity or Blender no problem.

Changing the material for a piece in Blender will change all of them.

[madjin]

Figured out texturing pre-generation, was simpler than I thought. Open the prototypes prefab then search for the piece you want to retexture. If that piece is missing from the Mesh Filter search for it again and respawn it. Then drag the material onto the piece and go back to the scene.

Next open ModuleData and click Create module data.

Can then re-open the Game.unity project and Initialize the area to generate with the remodeled / retextured pieces. Ta-da!

[madjin]

Author of the repo is working on a rewrite as a full blown city generator: twitter

How it looks with textures: https://pbs.twimg.com/media/E6B-OpGWYAIQeUJ?format=jpg&name=4096x4096 The new generator uses octrees and has these advantages compared to the old one:

• no generation failures (always succeeds) and no backtracking
• better sense of scale and more vertical structures
• 100% deterministic
• can generate LODs
• parallelizable

Hasn't decided if it'll be hosted on GH again. Think we'll be able to use the same assets though!

[avaer]

Sounds like we should proceed with generating Webaversey blocks then, but I think first it will need a lot more ingredients from Pinterest.

[madjin]

Notes on workflow

1. First we're starting with the base mesh layer. We could add simple textures at this step. There's 454 modules.
2. Next we can do a detail pass with an asset kit ready for adding plants / decorations. Doing this collaboratively or as a timelapse would make great fodder for social media.

When redesigning modular pieces, we'll want to edit the blocks.fbx file in the way I described at the top of the comment above:

Search for the piece you want to edit, then hide everything else to make it easier to work on just that. Make sure whatever you're doing is joined to that specific piece.

In order to check how a piece looks like next to another piece we can view previews in the Prototypes.unity file or generate everything and check. We'll want a quick feedback cycle. I think it might be fine to keep the UVs really simple, could retexture in post plus it could open up room for graffiti in the future.

Window Box

Name: High_Wall_Window Add window till for plants to overhang (don't add plants yet)

Now in the blend file you could search High Wall Window and find all the windows that need plants in the tills.

Interiors

Name: Interior_Wall

How might we improve the interiors? Note: the floor, wall, and ceiling are all part of the same modular section.

Example: Roofs

Name: Search "roof" Replace some with steel/glass to test

One of the common threads of solarpunk aesthetic is light coming in from high windows / glass ceilings. There is access to the rooftops, but the rooftops do not serve the interior sections well to allow for light.

One of the challenges with this is that there are a lot of individual roof pieces. Something we could do is perhaps instead of shingles, we replace some roofs with steel and glass and test.

Asset Kits:

After the initial base is generated we can go back in and add details

• doors
• lights
• plants
• signs
• booths
• vending machines
• fans (adds movement)

Plants

Signs and hanging decorations

Idea: Facade

The output leaves a lot exposed, so we'll need to think of how we want to patch that up.

One way is to add scaffolding on the outside and make it look like a work in progress. This would be an honest expression of how the world is a constant work in progress, similar to booming cities around the world. It's also kind of cool in the way that it's like the grid being applied vertically.

Another idea I had was to enclose it up, perhaps as part of a larger structure such as a mountain hillside. Gerudo Fortress was a cool example of such.

This would add a ton of cohesion since it'd be 1 model gluing everything together, make such optimized through by being enclosed, and be made pretty quickly since the procgen is very geometric. The rooftops would be unaffected.

[marian42]

Hello, I hope you don't mind me commenting here!

Some thoughts on the current blocks:

• I've started to put them in multiple files, that makes them easier to edit and it works better with version control. I use one file for roof blocks, one for interior blocks, etc
• The scale is a bit off, the blocks are 2 meters in size, but I think it would work better if they were scaled to 4 meters. Just keep in mind that you might need to scale them up at some point if you want to put regular sized humans into the map. And I think the railings are too tall relatively, so they might need to be scaled down.
• You can add your own blocks and set up data for them, this requires no code. It's totally feasible to add a new type of roof that lets light shine into interiors. Just keep in mind that there isn't necesarily an interior block behind every wall/roof, so I'd make variants of that new roof that can and cannot let light through. And of course you need a matching interior block that connects to the new roof.

Really looking forward to seeing progress on this, I love that you chose Solarpunk as the theme!

[avaer]

Some textures/shapes/colors ideas. This is Collision Chaos Good Future from Sonic CD.

[avaer]

An article that made the HN rounds: https://www.messynessychic.com/2021/06/10/inside-the-imaginarium-of-a-solarpunk-architect/

[madjin]

• Generate ideas
• Galad can do overpass art

[madjin]

Note: We have to replace each block individually in the unity project so its a good idea to take note of which pieces have been modified in the process.

[madjin]

Okay we figured out a better workflow, we transfer the modified .blend file of the blocks.fbx file and then export (with Apply Transform enabled) to overwrite the blocks.fbx file in the Unity project Assets folder. As a test @plankatron removed the columns from the wall and shingles on the roof and it worked during initialization.

[avaer]

The AI keeps coming up with Nihon City for this city (in Zone 0). Is that appropriate?

I googled it and it does not seem to exist so it is actually original (though I could be wrong).

I did not have any japan references in the prompt so if it came up with this multiple times it seems like it has a strong idea.

[avaer]

Checking in on the explanation of why it came up with that:

(this is AI generated)

Is Nihon City a good name for the name these characters inhabit? Why?

I think it is, and not just because I know the name Nihon means Japan in Japanese. I think it's a good name because it makes clear that the City isn't normal. But it is rooted in the real world. Also, I think it's because the name Nihon is easy to pronounce and remember, which is important when it comes to online video games. And I think the name is easy to say and remember because it's an actual name, not a made up one. The fact that it's a real name gives it a sense of authenticity.

[madjin]

Simplification

We need a more simplified generation in order to make concept art of new designs (geometry / textures) before modifying the pieces.

Modifications

1. Remove these lights, make these walls more plain.

2. Remove the Grandfather clock (perhaps we can consider a new prop instead?)

3. Simplify these walls by removing the lights

Updates

@plankatron figured out the minimal starter set for generating maps with: https://github.com/marian42/wavefunctioncollapse/issues/20, has 3 sets that work

[avaer]

@plankatron figured out the minimal starter set for generating maps with: marian42/wavefunctioncollapse#20, has 3 sets that work

But we are not doing that right? Only the full set.

[plankatron]

interesting shape language :

[madjin]

Those are some nice illustrations. I took some screenshots of the latest simplified blend, think we can concept art some good ideas on top of it. Here are some samples:

---

Note: If @plankatron you need front facing pics to make the UV maps faster we can use screenshots like this

Screenshots Imgur album: https://imgur.com/a/sq3naP7 GLTF export: https://cdn.discordapp.com/attachments/641036402829819914/867794958688976906/Map.zip

[avaer]

https://github.com/webaverse/app/issues/1365#issuecomment-884632571 interesting shape language :

I agree, very pretty and surreal. Like someone spent lifetimes on this one city. Which would be an appropriate vibe, though hard to accomplish without spending a lifetime. Luckily we have procgen :D

[madjin]

"his is probably the most accurate thing I've seen for how I envision city/Z0 looking like"

[madjin]

Workflow Speed Tip

You can edit the prefab inside Unity and nest objects inside modules then save the module data.

• open prefab
• drag fbx to module you want
• resize / reposition
• go to ModuleData and 'Create module data'

Then when you initialize the map it should start working in your editor

Can export as part of the gltf as well using plattar exporter. Saves a bunch of time!

[madjin]

[avaer]

☝️ Whoa, what's that?

It looks like it's neural.

[madjin]

Need to remove these tables

[GaladWarder]

☝️ Whoa, what's that?

It looks like it's neural.

It's the WIP I put in the Discord from yesterday - used deepdream to transfer the style quickly then I'm overpainting on top

[Vianvolaeus]

☝️ Whoa, what's that? It looks like it's neural.

I think I might have a way to achieve similar to this in SP 👀 the edge warping style is something i've messed with a little bit before... If you want me to explore it a bit, I can. It's generative (non-destructive) filtering.

[avaer]

It's the WIP I put in the Discord from yesterday

Thanks, I see it now.

[madjin]

Missed a spot here with the ceiling trim, need to simplify:

Need to remove the outdoor railings too: Roof_2_Wall Roof_2_Wall_M

---

Here's a preview on how it looks when initializing with nested assets:

https://user-images.githubusercontent.com/32600939/128097783-f86c99e7-e0f8-4cfe-8805-2a04f9e52f64.mp4

[madjin]

Comments / Ideas:

This area could be interesting for either hanging lanterns or scrolling LED text

We could make custom fit vines based on the mesh export of the city and edit in post for the columns:

Trees would look really cool at night with light decorations, would make the zone feel so much more comfy and safe

The trees outdoor need to be changed, they're not that great.

Overall this has interesting and sometimes surprising results that manual work might not provide the satisfaction of. I think there can be a balance between WFC and manual interior design.

[madjin]

Blender

6x6 area 12 textures 27.2 MB 58k triangles

https://cdn.discordapp.com/attachments/641036402829819914/872593885459775498/WFC_export.zip

[madjin]

Test file: https://cdn.discordapp.com/attachments/641036402829819914/873675931317858344/wfc_12x4.zip

Textures

I've begun testing out various textures for the pieces in blender by making quick and dirty uvmaps for the pieces and assigning CC0 textures using https://github.com/eliemichel/LilySurfaceScraper. I can rapidly test new textures since applying it to one piece applies to all the other pieces. For somethings like the roof there are 10 pieces.

I'm currently literally throwing stuff at the wall to see what sticks. Here's an unlit look at an example:

The style we want is cartoon / anime. I have ideas on how to post-process the chosen textures to fit the aesthetic.

---

Geometry

It would look nice to hang art on the walls. We need to smooth out the trim that is on some of these module pieces so we have clear wall space to hang some framed art.

Lights

I could add point lights to some of the module pieces and it will generate the map with all of them there. This is good for the props that have emissive lights, will save a lot of time.

When exporting to GLTF using plattar plugin and importing to Blender, the lights do appear as nodes but not as lights. Wondering if there's a way to search and replace in Unity or Blender?

[madjin]

Updated latest project files here: https://github.com/madjin/wavefunctioncollapse

[avaer]

https://github.com/webaverse/wavefunctioncollapse

[avaer]

Closing this because it’s not clear whether we can use the existing work, but the ingredient models are good. We still need to generate this.

[ahadshams]

reopening the issue and doing a deep dive

i found this and found the style quite interesting

https://manifold.garden/