[Spike] Design common noise models

[PersonTheCat]

Noise generation is central to almost every feature of Pangaea. We'll need to set up a very robust system for handling complex noise generators.

In vanilla, noise generation is modulated by DensityFunctions which can either wrap noise generators or transform output of other density functions. These are ultimately what create the terrain shape and influence generation of many features. In Cave Generator, we previously used a fork of FastNoise, which I intend to continue using in Pangaea.

It is my opinion that Pangaea should support but avoid density functions where possible. This is because density functions require an enormous number of wrappers and stack frames to perform simple computations.

We will want to design a handful of schemas that either generate FastNoise objects implementing DensityFunction or perform a series of computations designed specifically to produce terrain shape or some other purpose.

In the future, users will be able to provide real functions in their configurations, eliminating the need for most if not all nested wrappers in some cases.

I will update this description as I clear my thoughts and continue designing schemas in this thread. Please leave any relevant suggestions below.

[PersonTheCat]

Background

How this works may change slightly once XJS is ready, but in the meantime, we still start with DFU schemas.

In Cave Generator, we had one single NoiseSettings object which contained all settings needed for most noise generators. However, the implementation generated by these settings was a recursive data structure provided by our modular fork of FastNoise.

The schemas used by Pangaea will more closely follow the actual data structure of the generators, allowing much more complex configurations.

Additionally, we may decide to support wrapping Mojang noise generators in our FastNoise type. Either wrapping Mojang noise as FastNoise or simply using density functions wrapping either will do. TBD.

Basic Schemas

For starters, here's what a basic noise config might look like in DJS:

{
  type: 'simplex',
  frequency: 1.23,
  stretch: 1,
  offset: 0,
  range: [ -1, 1 ],
  invert: false,
}

This configuration--also valid in Cave Generator--produces a SimplexNoise generator with frequency of 1.23 and various other defaulted settings.

Complex Schemas

Unlike Cave Generator, however, we do not create complex types by adding more settings, but by updating our data structure.

For example, to produce fractal noise, we can create a FractalNoise generator, which exposes a few additional settings: octaves, lacunarity, and gain.

{
  type: 'fbm',
  lacunarity: 3.21,
  octaves: 3,
  gain: 0.5,
  reference: { 
    type: 'simplex',
    frequency: 1.23
  }
}

Thoughts on Complex Data Structure

This structure provides a kind of flexibility that just wasn't possible with the 1-dimensional schemas used by Cave Generator.

On the other hand, these generators may be a little harder to configure due to their increased size and depth. This is where XJS comes in.

With XJS, it will be possible to construct real functions that can be used as templates:

// types inferred by subsequent invocation
fn myNoise(type, frequency, octaves) {{
  type: 'fbm',
  octaves,
  reference: { type, frequency }
}}

myNoise {
  frequency: 3.21,
  octaves: 3
}

We can discuss additional schemas in the following comments

Edit: fixed a syntax error in fn myNoise...

[PersonTheCat]

Multi Noise

FastNoise currently provides 6 types of MultiNoise generators: min, max, avg, div, mul, and sum.

We can create these types in much the same way:

{
  type: 'sum',
  reference: [
    { type: 'simplex', frequency: 1.23 },
    { type: 'perlin', frequency: 3.21 },
  ]
}

We should also investigate adding a spline type, if not using Mojang's spline density function.

[PersonTheCat]

Warped Noise

FastNoise currently provides 3 types of DomainWarpedNoise generators: basic_grid, simplex2, and simplex2_reduced.

We can create these types in much the same way:

{
  type: 'basic_grid',
  amplitude: 4,
  frequency: 3.21,
  reference: [
    { type: 'simplex', frequency: 1.23 }
  ]
}

Note that we may need to rename a couple of these so as to not conflict with existing simplex types. Perhaps simplex_warp and simplex_warp_reduced will do for now, leaving basic_grid as is.

[PersonTheCat]

Noise Functions

How noise functions are implemented will ultimately be determined by XJS. It will be possible to reuse the current data structure (dispatching by type) and provide real type tokens in some locations, but I worry that the barrage of type tokens will be cumbersome to most users.

For now, the only way to implement this is using density functions:

{
  type: 'function',
  reference: {
    type: 'shifted_noise',
    noise: 'ridge',
    shift_x: 'shift_x',
    shift_y: 0.0,
    shift_z: 'shift_z',
    xz_scale: 0.25,
    y_scale: 0.0,
  }
}

In the future, perhaps we can continue supporting density functions in this location. How this is implemented will need to be discussed.

[PersonTheCat]

Supporting Real Functions with XJS

In order to support real functions with XJS, we have 3 choices:

1. continue using DFU + density function type
   • the user is required to specify that their function implements DensityFunction
   • we create a custom codec which checks if they gave us this type directly

{
  type: 'function',
  reference(x, y, z) { Math.random() } as DensityFunction,
}

2. continue using DFU + simple types
   • the user ought to specify the types of their function parameters
   • we create a custom codec which checks if they gave us an xjs.lang.Invocable
   • we use the function as an invocable, which auto-boxes and unboxes our inputs + may produce additional stack frames (not ideal)

{
  type: 'function',
  reference(x: number, y: number: z: number) Math.random()
}

3. switch to XJS schemas (which means we can't use these in any other DFU codecs)
   • instead of specifying the type string, the user provides a real type token
   • we use XJS schemas directly, bypassing DFU

FunctionNoise {
  reference(x, y, z) Math.random()
}

IMO 3 is the cleanest option. It looks like there is no way to completely avoid type tokens, but at least they would be used in a way that is very similar to dispatching by the type string property.

Which of these solutions we're able to use will both depend on and influence how we use XJS overall, so please discuss below.

Edit: to clarify: while I prefer reading and writing number 3, number 1 is the easiest solution by far and is the most compatible with our temporary setup before XJS gets released.

[PersonTheCat]

Real Life Use Cases for Noise Functions

Most likely, you will want to use noise functions to carefully manipulate the output of some other noise generator. Here's how that would look using solutions 1 and 3:

1. using DFU + density function type
   • other settings are just plain data, so you would need more code

const noise = pg.createNoise { 
  type: 'simplex', 
  frequency: 1.23,
}

{
  type: 'function',
  reference(x, y, z) { 
    noise.getNoise(x, y / 2, z) + x + z
  } as DensityFunction
}

3. switching to XJS schemas
   • since other fields use real types, the process is more consistent
   • requires less code

FunctionNoise {

  // actual generator, not data
  noise: SimplexNoise { frequency: 1.23 },

  // types are known and required by FunctionNoise
  reference(self, x, y, z) {
    self.noise.getNoise(x, y / 2, z) + x + z
  } 
}

Once again, I am largely in favor of solution number 3, but we do have to consider how incompatible it is with DFU. Perhaps we can support both options and, once again, simply use a custom codec which checks to see if real types were provided. I will continue to think about it.

[PersonTheCat]

A Compromise in Supporting Real Code

One compromise is to support both options 1 and 3. This gives us option 4.

4. support real types in DFU codecs && provide data-only codecs
   • we provide a codec which checks if the user gave us a real value or just data
   • the user can either provide type tokens to give us a real type or they can use the type string property.
   • most likely, we would have a preferred approach, that being to use type tokens
   • this does not use XJS schemas

Here's how that might look. Keep in mind that there must be a convention or official approach so as not to overwhelm users with conflicting solutions.

{
  // for this example, we mix both options, but users would just pick one
  type: 'sum', 
  references: [
    { type: 'simplex', frequency: 1.23 },
    SimplexNoise { frequency: 3.21 }, 
  ]
}

Pros:

• more flexible than 1 or 3 alone

Cons:

• we have to maintain real noise functions and complex codecs
• multiple ways to provide the same setup will be confusing to users who don't understand our systems
• users may see that they can refer to other noise generators directly and not understand when something is plain data or usable code

[PersonTheCat]

Since I'm working on this story now, I added TypedCodec to CatLib to support inlining code in normal codecs. I think that will work very nicely.

[PersonTheCat]

I have implemented codecs for FastNoise generators as a single codec. This is possible since the NoiseBuilder API in my fork of FastNoise already handles complex generator configurations.

I will mark this issue as resolved, but we are left with a few problems:

• Dispatching codecs by type would still be the more correct solution, as
  • it would allow us to control which fields are available contextually without needing output filters and extra validations
  • it would allow us to register additional codecs dynamically (e.g. from script or another mod)
• The current design uses a lot of lambdas, which Intellij struggles with. This is bad for maintenance.

If anyone reads this and would like to contribute, I will most likely accept PRs fixing either or both of the aforementioned problems. Thanks in advance! Otherwise, I will get to it as needed.