Request: Control perspective and lighting from the program

[p-e-w]

When rendering a Curv program, the viewer currently displays the shape using what looks like a standard perspective projection, slightly rotated, with a single white point light source located somewhere behind the camera.

I'd like to be able to control all of these aspects from the program itself. View parameters aren't strictly speaking properties of the shape, so it probably makes sense to group all of them in a view field.

Rough sketch of how this could look:

make_shape {
  ...
  view = {
    camera = {
      target = [0,0,0];  // Point the camera is aimed at
      direction = [1,0,0];  // Direction of the camera's location from the above point
      distance = 10;  // Distance of the camera's location from the above point
      zoom = 5;  // Magnification of the camera's field of view. Large values for both distance and zoom approximate a parallel projection.
    };
    lights = [
      {
        position = [5,0,0];
        color = [1,0,0];
        intensity = 50;
      },
      ...
    ];
  };
}

[doug-moen]

I like your proposal. @sebastien has asked for a related feature.

I don't want to put view parameters inside of a shape. As you say, they don't belong there, and it begs the question of whether the view parameters are preserved by transformations, and what happens if you union two shapes with different view parameters.

Instead, I am thinking of adding a new kind of graphical value, "view" values, which are peers to shape values. So, how about,

make_view {
   camera = {
      target = [0,0,0];  // Point the camera is aimed at
      direction = [1,0,0];  // Direction of the camera's location from the above point
      distance = 10;  // Distance of the camera's location from the above point
      zoom = 5;  // Magnification of the camera's field of view. Large values for both distance and zoom approximate a parallel projection.
    };
  lights = [
      {
        position = [5,0,0];
        color = [1,0,0];
        intensity = 50;
      },
    ];
  shape = cube;
}

It should be possible to make a parametric view (with sliders for view parameters), and a parametric view should also display value pickers for the shape, if the shape is parametric.

[sebastien]

So here are my two cents: in Curved, I have a .curved package format that contains the .curv source file as well as meta-data, which includes the camera controls. This makes it possible to store view-related presets along with the source code, while maintaining the distinction between both.

In general, and also in relation to #74, I think it would be ideal to have a way to override the fragment shader part with a custom curv function taking (shape,x,y,z,t) and returning the (r,g,b,a) pixel value. This makes it possible to do custom lighting, shading and camera control. Any parameter there (including camera controls) can be exported as a parametric value, and the editor/view can then store preset along with the source file.

I'm still in the early stages of integrating interactive parameter space exploration and preset saving, but it's pretty clear to me that you might want to save multiple presets for one Curv sketch, as the more parametric values you have, the more interesting configuration you might find.

The proposed make_view is interesting as a high-level construct that makes it easier to get started, but it should be built on a lower-level construct that does the fragment shader part, like what shadertoy is already doing.

[doug-moen]

At this point, all of the command line options for configuring the viewer window can now be specified using Curv code in the configuration file ~/.config/curv. Here is the current Viewer configuration:

• aa = supersampling factor for antialiasing, default is 1
• taa = supersampling factor for temporal antialiasing, default is 1
• fdur = frame duration for temporal antialiasing, default is 0.04 (25 FPS)
• bg = background colour, default is white
• ray_max_iter = max # of iterations in the ray marcher, default is 200
• ray_max_depth = max ray-marching distance, default is 400
• lazy = Redraw only on user input. Disables animation & FPS counter. Boolean, default is false

Based on discussion here, in issue #74, and in PR #75, here are some things I could do next:

• Add a Viewer config variable to select between the current standard render function (which is fast enough to use on my old Macbook Air), and Phillip's new render function (which is slower but produces much nicer output). Phillip and I would set this to different values in our config files. This could be called: renderer=#standard or renderer=#phong.
• Add additional Viewer config variables to control Phillip's new render function (specifically, lights).
• Add an optional material function to the Shape record, which is used by Phillip's render function.
  • Add a new kind of graphical value called a Scene, which is a record containing a shape and a collection of Viewer config variables.
• Add a new Viewer config variable, camera, which is useful in a Scene and on the command line. (OpenSCAD also has camera control variables on the command line.)

For example,

make_scene {
    shape = my_shape();
    renderer = #phong;
    lights = [...];
    camera = {...};
    bg = lib.web_colour.sky_blue;
}

[sebastien]

This is great! Will it be possible at some point to use curv directly to define a renderer?

[doug-moen]

@sebastien I think that defining a renderer in Curv will require extensions to the compiler to generate more appropriate GLSL code. It makes sense to do this work first. Consider it a stepping stone.

[doug-moen]

I added an optional render field to the Shape record. This contains rendering parameters, which at present can be any of the following:

• aa = supersampling factor for antialiasing, default is 1
• taa = supersampling factor for temporal antialiasing, default is 1
• fdur = frame duration for temporal antialiasing, default is 0.04 (25 FPS)
• bg = background colour, default is white
• ray_max_iter = max # of iterations in the ray marcher, default is 200
• ray_max_depth = max ray-marching distance, default is 400

The reason for putting this into the Shape record (as opposed to defining a new Scene type) is so that you can add rendering parameters to a public Shape project, without breaking other projects that might might be referencing it. Just think of the render field as metadata, which is ignored by code that uses the shape without rendering it.

My perspective is that Curv source files (*.curv) can be edited using an ordinary text editor and shared across the internet. At some point, I'll add a feature for referencing Curv source files and packages using URLs, instead of just local file system paths. The two common use cases are: a Curv source file defines a shape, or it defines a library. If a Curv source file defines a shape, then you'd like to be able to add rendering parameters without changing the type and breaking existing clients.

I acknowledge that the Curved project has a different design, and I'll have to discuss this with @sebastien. The code I just committed is not intended to be the final design.

This is a work in progress. Next steps:

• Add support for connecting any of the rendering parameters to a value picker using a parametric shape.
• Redesign the renderer and provide a new and better set of parameters. This means: the current render API is unstable.

[doug-moen]

There is a new render parameter called shader. It defaults to #standard (which is the old behaviour), but if you set it to #pew, then you get the shader that @p-e-w implemented in PR #75. Like all render parameters, you can specify it in configuration, on the command line, or in a Curv source file.

This parameter is just a temporary kludge to enable experimentation with @p-e-w's code, while I figure out an improved design.

[doug-moen]

You can now set the shader render parameter to {sf1: <function>}, which allows you to write a custom lighting function in Curv. Since this is still experimental code, I didn't know the best API to use, and I haven't figured out good names yet. sf1 stands for "shading function 1", so I could add another API sf2 later if it turns out we need more than one.

sf1 is only called during ray-casting if the ray hits the surface of a shape. The arguments are: sf1(pos, nor, rd, col).

• pos is the position, in 3D space, of the point on the surface of the shape that we are shading.
• nor is the normal to the surface at that position.
• rd is the ray direction.
• col is the colour at that position, computed by the shape's colour function.

sf1 returns a pixel colour. It's a modification of col based on the lighting model.

Here's an example that I tested. This lighting function is the same as the default lighting function, except that I removed ambient occlusion (which makes very little difference anyway, based on the models that I tested).

let
    reflect(I,N) = I - 2.0 * dot(N, I) * N;
in {
  ... cube;
  render: {
    shader = {sf1(pos,nor,rd,col) =
        let
            ref = reflect( rd, nor );
            lig = normalize(-0.4, 0.6, 0.7);
            amb = clamp( 0.5+0.5*nor[Z], 0, 1 );
            dif = clamp( dot( nor, lig ), 0, 1 );
            bac = clamp( dot( nor, normalize(-lig[X],lig[Y],0)), 0, 1 )
                * clamp(1-pos[Z], 0, 1);
            dom = smoothstep( -0.1, 0.1, ref[Z] );
            fre = clamp(1+dot(nor,rd), 0, 1) ^ 2;
            spe = clamp(dot(ref, lig), 0, 1) ^ 16;
            lin = 1.30*dif*(1.00,0.80,0.55)
                + 2.00*spe*(1.00,0.90,0.70)*dif
                + 0.40*amb*(0.40,0.60,1.00)
                + 0.50*dom*(0.40,0.60,1.00)
                + 0.50*bac*(0.35,0.35,0.35)
                + 0.25*fre*(1.00,1.00,1.00);
            iqcol = col * lin;
        in
            lerp(col, iqcol, 0.5);
    };
  };
}

[sebastien]

Nice, that's one more step in the direction of having shadertoy-like functionality!

[doug-moen]

Camera control needs to be a separate issue: #56.

Lighting control needs to be part of a larger issue, which is a new PBR (physically based rendering) system. PBR is a very complex topic, so I want existing open source code that, as much as possible, can simply be plugged in to Curv. I don't want to reinvent PBR from first principles.

Eg, we could clone the Eevee renderer used by Blender 2.80. The Eevee code supports the sophisticated rendering that people want, it's open source, it renders in real time, it uses OpenGL 3.3 (same as Curv), and the algorithm appears to be compatible with Curv's signed-distance-field representation. Blender has designed a set of PBR parameters which they call "Principled BSDF", which is compatible with Renderman and Unreal Engine.

https://docs.blender.org/manual/en/latest/render/shader_nodes/shader/principled.html