How to light a `Scene`?

[tomcrane]

Rob’s draft:

A Scene has an ambient light source, given in the ambientLight property, which if not present has a default of 0.75 intensity white light. Other lights can be added in a model file, and annotated on to the Scene. This allows the simplest case of just ambient white light to be assumed, and gives publishers full control over lighting in more complex cases. Lights which are in other models MUST be rendered only if the Annotation of the model has the "render-light" behavior.

Question How to do "headlight" lights that are tied to a camera, when that camera can move or be swapped away from to a different camera? Need to turn lights on or off?

The Scene has an ambientLighting property:

"ambientLighting": {
    "id": "https://example.org/lights/1",
    "type": "Lighting",
    "color": 256,
    "intensity": 1.0
}

If a Scene does not have an ambientLighting property, then the client SHOULD assume an ambient light with white color and 0.75 intensity.

[azaroth42]

Propose that headlights should be a separate issue from ambientLighting

[mikeapp]

We might also want to investigate scene-level directional light (https://threejs.org/docs/#api/en/lights/DirectionalLight):

"This light will behave as though it is infinitely far away and the rays produced from it are all parallel. The common use case for this is to simulate daylight; the sun is far enough away that its position can be considered to be infinite, and all light rays coming from it are parallel."

[tomcrane]

This may be many distinct issues covering:

• ambient light
• headlights
• lights included in models
• lights defined independently of models (scene-level), positioned like #2257 (cameras)

...and if the latter is in scope is it just a IIIF-isation of what threeJS can do?

We should take care in considering implementability for whether these are in scope.

[JulieWinchester]

At the Basel working meeting on October 26, 2023, after some back-and-forth and a few different iterations of possible structures, general consensus emerged for the following:

• Lights should be supported as annotations. This includes ambient lighting, and there would be no lights at the Scene property level.
• A type property on the lighting annotation should allow for differentiating between for types of lights: ambient, directional, point, and spotlight.
• There should be a new DirectionTransform allowed for lights (and probably cameras) to specify orientation/directional-facing for an element that needs such facing, like a light or a camera. Not all lights would support a DirectionTransform (see table below). To specify orientiation, DirectionTransform should take either a unit-length vector or a Scene reference target combined with a PointSelector. In the latter case, this PointSelector location specifies the "look-at" target of the light, and the orientation would have to be calculated from a combination of the light's position within the scene and the position of the look-at target.
• Some lights should be able to specify lighting position through target PointSelector. Not all lights will accept a target position, though (see table below).
• All lights likely should take color, intensity, and time duration properties. Some specific light types will need further properties. Point type lights may want distance or decay properties. Spotlight lights may want the same properties as point lights but also including an angle property.

Light type	Direction (specified in body)	Position (specified in target)
Ambient	NO	NO
Directional	YES	NO
Point	NO	YES
Spotlight	YES	YES

[tomcrane]

In the table above, are the different lights all of type Light with different behavior, or are they different types: AmbientLight, DirectionalLight etc. ?

[tomcrane]

In Washington DC:

There are many types of light and this is an area of rapid change in 3D. E.g., functional lighting Image-based lighting, lots of other types of lights (examples needed).

Should IIIF include lighting in the core spec at all? Just let viewers provide their own lights. (This might be messy when combining models into a Scene, you might really need light to make Scenes usable)

Should IIIF provide some components like DirectionTransform but not specify lighting types?

Are lights all extensions in IIIF?

(NB Image-based lighting is quite easy to define in IIIF terms, just position the image in the scene)

If we don't define IIIF lights, there's a barrier to lighting your scene - could IIIF host light models for you? This might be impractical, inappropriate intensity values etc

Consensus - we should support simple lights in spec and make it easy to introduce new lighting concepts through extensions that can evolve more rapidly.

We should implement the four types in the table above in the spec.

But - lights are OPTIONAL and viewers should expect to deal with IIIF with no lights, models with no lights, and be able to provide lights themselves.

Should all lights be in extensions so developers can see them all in the same place?

(related - ability to disable lighting on painted models)