Move world genesis, terraforming and processing to separated thread/task_pool

[afonsolage]

Currently the chunk generation, loading and neighborhood calculation is done on genesis stage of pipeline, but there are other heavy processing tasks, like stated on #7. So in order to solve this, it would be better to move all chunk generation, processing and everything until the final VertexFace and cache it.

Overview

The idea is to store the processed data in sectors. Each one will have 16^3 chunks (see #10 ) and will be stored in its own file. So there are three flows possible:

1. Sector Exists
   1. Load;
   2. Send event;
2. Sector Doesn't exists
   1. Generate kinds;
   2. Process Pipeline;
      1. Get Neighborhood;
      2. Faces Occlusion;
      3. Faces Merging;
      4. Vertex Computation;
      5. Save;
      6. Send event;
3. Sector Updated (or one of its chunks)
   1. Update chunk;
   2. Process Pipeline (2.ii);

The general hierarchy will be world -> sector -> chunk -> voxel, where:

• world controls the loading, unloading and updating of sectors;
• sector holds and updates all chunks all once;
• chunk holds all voxels;
• voxel a single unit value inside a chunk

Only a single sector will be updated/processed at a single time, due to it's size.

landscape will query world to get the desired number of chunks as it wants in order to send to rendering pipeline. This will be determined by a draw distance.

The load/unload process of sectors should be triggered when player moves at half way of the current sector.

Drawbacks

1. Memory usage can be pretty high, if each voxel uses 5 bytes (2 bytes for kind, 1 byte for light, 1 byte for faces occlusion and 1 byte for other data), it will consume 516^316^3*9^3 = 2.1 GB. This can be reduced by using compression, since many voxels will be empty or repeated.

2. When adding propagation, which we currently doesn't have, the game may slowdown if the propagation happens between more than 1 sector, like light or fire spreading, the worst case scenario can be an edge between 8 sectors, which may take a while to process. This is a corner case and shouldn't be a big problem at all

3. When a player move quickly and triggers the loading and unloading of sectors, moving back and forth, this may slowdown the game, due the high amount of sectors being loaded/unloaded.

4. Needs to test the loading/unloading of sectors on slow HDDs and check how it behaves. Currently I'm using M2 SSD, which is very fast and should not be considered a normal use case.

[afonsolage]

This must address also #19

[afonsolage]

Some thoughts on this subject:

Use Bevy Sub App

• :heavy_plus_sign: keeps ECS code consistent
• :heavy_plus_sign: easier to reason about
• :heavy_minus_sign: Not seems to be stable
• :heavy_minus_sign: Low documentation, needs to read src to understand how it works
• :heavy_minus_sign: Needs to sync two app (main and sub app)

Use Task Pool :heavy_check_mark:

• :heavy_plus_sign: Better performance due to fully parallelization of heavy work
• :heavy_plus_sign: Stable, even tho there is a change coming in 0.8, the current method still works (I belive)
• :heavy_minus_sign: Add boilerplate to keep track of tasks completed

[afonsolage]

Currently there is a single pipeline which covers from generating chunks, cache, computing vertices, occlusion, merging faces, [..] until the rendering.

We should split this work and have the following setup:

Terraformation

• Unload chunks;
• Load chunks;
• Generate and save existing caches;
• Process terraformation pipeline;
  • Generate voxel kind;
  • Compute neighborhood;
  • Faces Occlusion;
  • Faces Merging;
  • Compute Vertices;
• Must have a configurable working radius, computed from a marked entity transform position;

Rendering

• Despawn entities;
• Spawn entities;
• Add render related components
  • Material
  • Mesh
• Must have a configurable working radius, equals or less than Terraformation, computed from a marked entity transform position

Unresolved Questions

• Consider moving terraformation to another crate, since it would be a nice idea to have decoupling between Terraformation and Rendering, also it would be possible to enable optimizations for Terraformation crate while debugging Rendering
• Since Terraformation will have likely a higher working radius than Rendering the memory consumption should increase drastically, so it may need some work to reduce memory usage, like in-memory compression or some heavy cache utilization.
• Is the sector concept introduced on this issue still relevant? I don't think so