instead of leaf nodes being voxel, they should be indices to a NxNxN collection of voxels.

A N value of 4 is probably a good mix between big enough while still not consuming too much extra memory, but it will have to be benchmarked https://dspace.library.uu.nl/handle/1874/315917

Pros

faster editing in general
working with neighbors should be fast as long as it is within the NxNxN chunk
- faster bounces since we can check if we are still in the local grid before doing a full octree lookup!
- faster shadow rays when light is local to chunk
since we implicitly know that each entry is a leaf voxel, the byte size can be kept small. I.e: one row can be represented with a two u32 that represent 4 byte material indices each using glsl's bitfieldExtract()
traversal of occupied areas will be faster since a neighbor lookup does not have to go through the octree

Cons:

struct Chunk {
 // each voxel is a 8bit uint which is the voxel's material index
 [4][4]uint row
}

const Chunk = packed struct {
  row: [4][4]packed struct{
    x: u8,
    y: u8,
    z: u8,
    w: u8,
  }
}