search

michael-fadely / sadx-dc-lighting

Sonic Adventure 1 palette lighting for SADX PC.
MIT License
39 stars 3 forks source link

Secondary lighting system for non-palettized objects (SL files) #22

Open PiKeyAr opened 7 years ago

PiKeyAr commented 7 years ago

This is an interesting effect, understanding which may help shed some light on the purpose of the floats found in SL files. It appears that setting the following float to 0 makes hint monitors pink: float It's 3F 80 in most cases. I have seen 3F 30 (SL_W5B.BIN) but it doesn't seem to make the monitors pink. Interestingly none of the data found after 0x600 seems to have any effect on this.

PiKeyAr commented 7 years ago

So it appears that immediately after light direction (first 8 bytes) there are 3 floats (R G B - in that order) affecting diffuse color of some objects, kinda like Ambient RGB? The first series of RGB values affects only the hint monitors and the button (O SWITCH). btn Very interesting.

michael-fadely commented 7 years ago

Looks like lighting parameters for objects that aren't palletized for whatever reason, which seems to consist of the switch and the hint monitor.

PiKeyAr commented 7 years ago

List of objects that use this system in SA1 with their corresponding display functions in SADX: Common switch (4CB590) Hint monitor (7A9460) OSandSwitch (53EA50) OEggKanban (5B5100) Sonic's snowboard (494140) Tails' snowboard (45BC40) Master Emerald shards (4A2C70) - SL file RGB values ignored in SA1 Froggy in Gamma's Emerald Coast (4FA320) - SL file RGB values ignored in SA1 E101 Beta's rocket (5683A0) Shadow blob (58C740) - uses SL but it isn't visible under normal circumstances because the texture is black Chaos puddle (w/environment map) in several cutscenes: after Egg Hornet (6EDE00), Knuckles' intro, Big's intro (likely all the same model/function) Chambr in Sky Deck (5F6E90) Various debug objects such as NoWater and Ototto's collision Hedgehog Hammer targets (528890 and 626770) Chao - SL file RGB values ignored in SA1, textured (silver, gold etc.) Chao use palettes

PiKeyAr commented 6 years ago

So uh... Apparently the SL objects are slightly bugged in SA1, which may have interfered with our previous tests. If you load an Action Stage right after booting up the game, the hint monitor in Twinkle Park looks like this: image However, if you die or restart the level, it looks like this: image Considering how the hint monitor's shadow disappears after a restart too, I think the first screenshot has the "correct" lighting, while the second screenshot shows different lighting caused by some bug.

PiKeyAr commented 6 years ago

I've been looking at the last section in SL files starting at the offset 0x780. So far I found out that this section can hold various types of data: 1) Another set of values for light direction and ambient color stuff. For example, SL_X0B.BIN (MR day) has some different values in that area. It matches the data in SL_T1B, which is totally unrelated, so it may be a leftover of some "default" light direction/ambient light setting? Or it could be just random garbage. 2) A pair of diffuse/specular 16-color palettes. The colors are often identical to those in the corresponding PL file's first pair of palettes (SL_82B). In at least one case they're followed by another pair of palettes that consists of 64 colors (SL_J0B.BIN), which are also the same as the second palette pair in the PL file. 3) Nothing. The file can have this section consist of 00, or not have this section at all.

The above leads me to believe that the latter sections of the SL file are filled with random data that is unlikely to be used in a meaningful way.

PiKeyAr commented 5 years ago

OK, so... With some help from ItsEasyActually, who provided a 1ST_READ disassembly, I was finally able to find that evasive data for the hardcoded second light. Also I forgot to say but I tested my hypothesis a while ago and came to the conclusion that these lights aren't using generated palettes, but that's probably obvious now.

Here's my current understanding of the SL system. There are two (possibly more, but I haven't seen more than two) lights set up for "SL objects":

"Directional light" (DirLight) This is the one with hardcoded light direction. The light direction is set up in the level binary, but there's also a default light direction in 1ST_READ. The light has the following (all hardcoded) parameters:

"Environment light" (ENV) This is the "normal" SL light that we figured out earlier.

Getting the data for the hardcoded light from SA1 binaries is tedious, but I made a table for Cheat Engine that lets you check the values for both DirLight and ENV. The "current" values can be changed in realtime. This works on a 1.001 CDI (US with static title screen). There's also a list of additional "Free/Slave" values related to lighting, but so far I haven't seen them used in any meaningful way.

Looks like this is good enough to try in Lantern Engine now?

PiKeyAr commented 5 years ago

Here's the DirLight data I got from SA1 levels. I rounded almost all floats to make it easier to read. There's a set of values that is used almost everywhere, and several levels override it.

struct DirLightData
{
    char LevelID;
    char Act;
    NJS_VECTOR LightDirection;
    float R;
    float G;
    float B;
    float Specular;
    float Diffuse;
    float Ambient;
};

DirLightData DefaultDirLight = { 0, 0, { 0.822f, -0.57f, 0 }, 1, 1, 1, 0.8f, 1, 0.25f };

DirLightData DirLightOverrides[] = {
    {LevelIDs_TwinklePark, 0, { 0.5f, -0.866, 0 }, 1, 1, 0.95f, 0.8f, 0.9f, 0.2f }, //same for all acts
    {LevelIDs_SpeedHighway, 0, { 0.5f, -0.866, 0 }, 1, 1, 1, 0.7f, 0.6f, 0.2f },
    {LevelIDs_SpeedHighway, 1, { 1, 0, 0 }, 1, 0.9f, 0.9f, 1, 0.8f, 0.3f },
    {LevelIDs_SpeedHighway, 2, { 0.866, -0.5f, 0 }, 1, 0.9f, 0.9f, 0.65f, 0.8f, 0.35f },
    {LevelIDs_LostWorld, 0, { 0.6785513163f, -0.7345529795f, 0 }, 1, 1, 1, 0.8f, 0.4f, 0.05f },
    {LevelIDs_LostWorld, 1, { 0.866, -0.5f, 0 }, 1, 0.9f, 0.9f, 0.65f, 0.8f, 0.25f }, //act 3 same as act 2
    {LevelIDs_IceCap, 0, { 0, -1, 0 }, 1, 1, 1, 1, 1, 0.36f },
    {LevelIDs_IceCap, 1, { 0, -1, 0 }, 0.59f, 0.75f, 0.77f, 1, 1, 0.22f },
    {LevelIDs_IceCap, 2, { 1, 0, 0 }, 1, 1, 1, 1, 1, 0.68f },
    {LevelIDs_IceCap, 3, { 0, -1, 0 }, 0.59f, 0.75f, 0.77f, 1, 1, 0.22f },
    {LevelIDs_PerfectChaos, 0, { 0.5f, -0.866f, 0 }, 1, 1, 0.95f, 0.8f, 0.9f, 0.2f }, //act 2 same as act 1
    {LevelIDs_SandHill, 0, { 0, -1, 0 }, 1, 1, 1, 1, 1, 0.36f },
    {LevelIDs_TwinkleCircuit, 0, { 0.5f, -0.866f, 0 }, 1, 1, 0.95f, 0.8f, 0.9f, 0.2f },
    {LevelIDs_StationSquare, 0, { 0.5f, -0.866f, 0 }, 1, 1, 0.95f, 0.8f, 0.9f, 0.5f }, //same for all acts
    {LevelIDs_Past, 0, { 0.5f, -0.866f, 0 }, 1, 1, 0.95f, 0.8f, 0.9f, 0.5f }, //same for all acts (I guess)
};

Some notes on the similarities between the SL system data in SA1 and Stage Lights in SADX:

Here are some discrepancies that I can't explain yet:

FINAL EDIT I PROMISE (OK I lied) Remember the weird "CharacterLightData" stuff in SADX disassembly at 0x9008E4? With the only two exceptions being Sky Deck and Hedgehog Hammer, the array lists the same levels that override DirLight in SA1. There's some code in SADX that copies this data into an otherwise unreferenced "CurrentCharacterLight", other than that it's completely unused. I think the layout of it in the disassembly is incorrect though. Here's what I think is the correct structure:

struct CharacterLightData
{
  char LevelID;
  char Act;
  char LightIndex;
  char UseDirection;
  NJS_VECTOR LightDirection;
  float Specular;
  float Diffuse;
  float R;
  float G;
  float B;
  float AmbientR;
  float AmbientG;
  float AmbientB;
};

Also the array starts at 9008E8, not 9008E4, even though the code is pointing to 9008E4. I think this may have caused issues with defining it in the past. Anyway this looks even more similar to DirLight stuff than the Stage Lights.

PiKeyAr commented 5 years ago

I think I found out how the game determines (at least partially) which objects use non-palettized lighting. It's the objects whose models/objects/actions/motions are drawn using these functions: 406FA0 - draws an NJS_OBJECT with NJS_MOTION 406FE0 - draws an NJS_OBJECT with two NJS_MOTIONs (not sure what exactly this is but whatever) 4084F0 - draws an NJS_MODEL 40A100 - draws an NJS_OBJECT All of the above functions do this before rendering the models: njSetConstantMaterial(&DefaultSpriteColor); Here's a list of objects drawn using these functions:

This covers a lot of SL objects found so far, and I don't think it's a coincidence because no other objects call those functions. However, several other known SL objects (notably the hint monitor, common switch, OEggKanban etc.) don't call those functions in SADX, so either their code was changed or that isn't the only factor.

michael-fadely commented 5 years ago

Here's what I think is the correct structure:

struct CharacterLightData
{
  char LevelID;
  char Act;
  char LightIndex;
  char UseDirection;
  NJS_VECTOR LightDirection;
  float Specular;
  float Diffuse;
  float R;
  float G;
  float B;
  float AmbientR;
  float AmbientG;
  float AmbientB;
};

Also the array starts at 9008E8, not 9008E4, even though the code is pointing to 9008E4. I think this may have caused issues with defining it in the past. Anyway this looks even more similar to DirLight stuff than the Stage Lights.

This doesn't quite add up. It uses 9008E4 as a source address for a memcpy, which really shouldn't be happening if it's offset. It does make more sense as far as the data itself is concerned, though, so I'm conflicted here.

Edit: I believe the one at 9008E4 is a default-initialized global variable, and the actual array of data starts at 00900918.

michael-fadely commented 5 years ago 
* The Stage Lights structure has a "use direction" flag, which sounds like it could be used to distinguish between DirLight and ENV light direction (e.g. flag enabled = use the hardcoded light direction instead of the stage one). In SADX Stage Lights there isn't a single light with this flag off, though.

There are actually several with that flag off, but they get skipped by Direct3D_PerformLighting.

PiKeyAr commented 5 years ago

Found another SL object (black in the screenshot): image SA1 model at 0015F77C in STG05.BIN Curious details: material exponent set to 0. SADX rendering code: sub_60BB60.

The unused AokiSwitch object in Lost World is also an SL object. Its display sub in SADX is at 5E66D0 and it looks old enough to have been unaltered from the DC original. It does the following with njControl3D manipulations, though I think the parts rendered before this are also using SL lighting: njControl3D(NJD_CONTROL_3D_MODEL_CLIP | NJD_CONTROL_3D_DEPTH_QUEUE | NJD_CONTROL_3D_OFFSET_MATERIAL | NJD_CONTROL_3D_ENABLE_ALPHA | NJD_CONTROL_3D_CONSTANT_ATTR);

PiKeyAr commented 4 years ago

@Exant64 and I have been working on figuring out non-palettized lights in SA1, and thanks to his research we're getting very close to resolving the entire puzzle. Much like in SADX, there are different kinds of "render a model" functions in SA1. Depending on the variation of the model rendering function it uses a different lighting setup. In our current understanding of the SA1 lighting system, there are at least four kinds of lighting used in the game:

List of known models that use these kinds of lighting: Normal (two Ninja lights w/color): common switch, hint monitor, OEggKanban, Chaos puddle in some cutscenes, Aoki Switch (top) Easy (one non-Ninja light w/color): Sonic's snowboard, Tails' snowboard, OSandSwitch, shadow blob, Chambr in Sky Deck, debug objects, Aoki Switch (bottom), E101's missiles, Hedgehog Hammer targets Simple (one Ninja light w/o color): rocks in Red Mountain, Chao (non-textured), Master Emerald pieces, Froggy in Gamma's Emerald Coast

Functions drawing non-palettized models in SADX - this is only a rough estimate because a lot of objects use different model draw functions in SADX, and as a result some stuff matches and some doesn't. Two lights: DrawModel_ResetRenderFlags Easy light: 4084F0 (MODEL), 40A100 (OBJECT), 406FA0 (MOTION), 405EF0 (two MOTIONs maybe?) - lines up almost perfectly with SA1 except the Chao Name Machine, which uses palettes in SA1 Simple light: The "simple light" objects share the model rendering function with a lot of other stuff in SADX, so this might not be it. 407A00 is my best bet. The rocks in Red Mountain and Froggy use this, Master Emerald shards don't (because of queuing I guess), and the Chao use chunk models in SADX so they obviously don't either.

Now about the lights involved:

Two lights mode, first NJS_LIGHT (SL/ENV light) Light direction (njSetLightDirection): same as palette (0, -1, 0 vector NjRotated using Y and Z values from the SL file) Light color (njSetLightColor): R G B from the SL file Light intensity (njSetLightIntensity spc, dif, amb): from the SL file

Two lights mode, second NJS_LIGHT (DIR light) Light direction: it makes an NJS_VECTOR by taking two floats (usually hardcoded in the stage binary) as X and Y, adding Z=0 and njUnitVectoring it Light color (njSetLightColor): called in a function called by LoadLevel, also called in stage load code in some levels Light intensity (njSetLightIntensity spc, dif, amb): same as above

Easy light Light direction (njSetEasyLight) - same as palette Light color (njSetEasyLightColor) - R G B from the SL file Light intensity (njSetEasyLightIntensity dif, amb) - from the SL file, no specular

Simple light Light direction (njSetLightDirection): same as palette Light color (njSetLightColor): called when a stage loads, never seen it set to anything other than white Light intensity (njSetLightIntensity spc, dif, amb): from the SL file

michael-fadely commented 4 years ago 
* "Two lights" mode - non-palettized lighting that uses two Ninja lights. One is set up using the parameters in the SL file (SL light/ENV light), and the other is set up in code, usually in level binaries (DIR light). The ENV light has one configurable color that is shared for diffuse and specular, the DIR light can have different diffuse and specular color (but I think the game doesn't use that). Called by njDrawModel/Object. Ninja lights are just that - they use NJS_LIGHT structures described in the Katana SDK.

Are the parameters that are set up in code embedded in the instructions, or are they stored as proper data that we can extract? This will be vital for getting the right look.

PiKeyAr commented 4 years ago

I made a simple Dreamcast program to preview lighting based on what we know so far. I'll import the same model in SADX and try to set up a similar environment to make it easier to test lighting. image

EDIT: And here's a stubbed version of this as a mod for SADXPC. Functionality should be identical except lighting.

PiKeyAr commented 1 year ago

After looking at this system again with various bits of info pieced together from SA1 DC behavior on the emulator and SADX symbols, I think it's finally feasible to attempt to imitate it. I'll put this here for reference in case we end up implementing it in the mod.

In SA1, most models were rendered with palette lighting using the game's custom function, but some models were rendered by calling Ninja drawing functions directly, which made them use the Ninja lights system. There are three light setups depending on the specific variation of the Ninja draw function:

Light setup Functions in SADX Comment
Regular njDrawModel (0077EF70) Allows to use multiple NJS_LIGHTs of various types (directional, point, spot). Specular is supported.
Simple ___SAnjSimpleDrawModel (00408520), __SAnjSimpleDrawObject (0040A130) Only the first registered NJS_LIGHT can be used. It is set up as a directional light, and intensity and color settings cannot be adjusted. Specular is supported.
Easy njEasyDrawObject (0040A100), njEasyDrawModel (004084F0), njEasyDrawMotion (00406FA0), njEasyDrawShapeMotion (00406FE0) Uses one "easy" light (not NJS_LIGHT). Specular is not supported. From the SDK: The light vector uses the screen space coordinate system. E.g. if set up like njSetEasyLight( 0.0f, 0.0f, 1.0f );, the light source will always be in front of the screen. To set the light source in 3D space, they must be converted with the current matrix: light.x = LIGHT3D_X; light.y = LIGHT3D_Y; light.z = LIGHT3D_Z; njCalcVector( NULL, &light, &lwork ); njSetEasyLight( lwork.x, lwork.y, lworkz );After this, the Easy light source will be at LIGHT3D_X,LIGHT3D_Y,LIGHT3D_Z

The SADX functions are a very rough match since SADX changed draw functions for a lot of objects, and all that is left is some wrapper functions that call a more generic draw function: 1) The Regular functions are difficult to tell from the paletted functions, so the above is just my best guess because it's called directly by the switch and the AokiSwitch. 2) The Easy functions are mostly still there. 3) The Simple functions only have two stubs which aren't called by anything other than the Chao Name Machine, which doesn't match SA1.

Here are the data sources for the lights described above:

Light Type Direction Color Diffuse, specular, ambient intensity
Regular - light 1 Directional (0, -1, 0) rotated using RY and RZ values from the SL file From the SL file From the SL file
Regular - light 2 Directional njSetLightDirection called in the skybox display function njSetLightColor called in the skybox display function njSetLightIntensity called in the skybox display function
Simple Directional (0, -1, 0) rotated using RY and RZ values from the SL file White (in SA1 anyway) From the SL file
Easy Directional (0, -1, 0) rotated using RY and RZ values from the SL file From the SL file From the SL file, no specular

In addition to these lights, there can be custom lights set up by objects' code, which will affect all objects rendered with Regular functions. These custom lights can be directional, spot or point, with or without specular. The custom lights are created with the function le_CreateLight (0040A510 in SADX) and deleted with the function le_DeleteLight (0040A550 in SADX).

In the Dreamcast version, these functions use pointers to actual NJS_LIGHTs. In SADX, the Ninja lights system appears to have been replaced by a system using the structure LE_LIGHT_ENV (what we know as "Stage lights"). Instead of specific pointers, there is now an array of four "current stage lights" called le_env at 03ABD9F8. It also has a corresponding array of four Ninja light flags le_env_typ at 03ABDBB8. So instead of NJS_LIGHT pointers, the light create/delete functions manipulate data in these arrays in SADX.

Interestingly enough, the data for the light "Regular- light 2" is completely the same in SADX. It's still set up by the skybox display functions which call njSetLightColor etc. with the same arguments as the Dreamcast version, so basically we don't need the data from this post. However, now that these functions are redirected to modify the first entry in the le_env array, this data is overwritten every time a stage loads. That is happening because the function le_set_env (0040A95) copies over all light data from the SADX stage lights array (00900E88), which happens every time a light type is set and whenever a new level loads.

I found the following behavior of Ninja lights different from palette lighting: NJD_FLAG_IGNORE_SPECULAR disables specular lighting completely per material. NJD_FLAG_USE_FLAT enables flat shading (used by Master Emerald pieces).

Custom lights

There's a leftover function in SADX that creates custom lights associated with specific objects: asSetLight at 0040ADC0. It uses a taskwk pointer, an NJS_LIGHT pointer and a pointer to a structure called LightInfo:

struct LightInfo
{
  int lsrc; // Ninja light flags
  float spc;
  float dif;
  float amb;
  NJS_ARGB argb;
  NJS_POINT3 pos; // Light position (for spot and point lights)
  NJS_POINT3 vec; // Light direction
  float nrang; // Distance at which light source attenuation begins
  float frang; // Distance to cut the effect of the light source (range)
  int iang; // The angle at which the specular or spotlight is at maximum intensity
  int oang; // The angle at which the specular or spotlight effect ceases
};

The following Ninja light flags can be used. I've bolded the ones that I've seen actually used, and crossed out the ones that are unlikely to have been used in SA1.

Flags Flag name Light type Ambient Diffuse Specular
0x01 NJD_AMBIENT Ambient 〇 × ×
0x02 NJD_DIR_LIGHT Directional × 〇 ×
0x03 NJD_LAMBERT_DIR Directional 〇 〇 ×
0x04 NJD_POINT_LIGHT Point × 〇 ×
0x05 NJD_LAMBERT_POINT Point 〇 〇 ×
0x08 NJD_SPOT_LIGHT Spot × 〇 ×
0x10 NJD_SPEC_DIR Directional × 〇 〇
0x13 NJD_PHONG_DIR Directional 〇 〇 〇
0x20 NJD_SPEC_POINT Point × 〇 〇
0x25 NJD_PHONG_POINT Point 〇 〇 〇
0x40 NJD_USER_LIGHT Call a custom callback function
0x80 NJD_SIMPLE_LIGHT Simplified lighting calculations (not the same as the "Simple light" described earlier)
0xC0 NJD_BLOCK_LIGHT Simplified lighting calculations

The following custom lights have some leftover code and data in SADX. These also exist in SA1 but they aren't visible normally because the models that were supposed to be lit by them are rendered with palette lighting. 1) Ice Cap background light: NJD_PHONG_POINT, LightInfo structure at 00C67FA0. Uses angles. 2) "Chaos underlight": NJD_POINT_LIGHT, NJS_LIGHT structure at 03D0D5B8. The light is initialized using the function ChaosUnderLightInit (007AD250) that has a "radius" argument which goes unused in SADX but in SA1 it should be the light range. In Chaos' main functions, the light's matrix is transformed to match Chaos' position. 3) Some light in Knuckles' story cutscene: NJD_SIMPLE_LIGHT | NJD_LAMBERT_DIR, function at 006868F0. 3) Lights in Gamma's briefing cutscene: NJD_SIMPLE_LIGHT | NJD_LAMBERT_DIR, function at 006F4570. The light is a leftover, but this effect was adapted to palette lighting, and palette-related functions are called here too.

Specular

The Ninja lights use some kind of special specular calculation. I think this is going to be the main difficulty in recreating this system. Exant has reversed the specular light formulas for Simple and Easy lights (for chunk models though), maybe it's the same for basic models.

image

i dont remember which was easy and which was simple

one of them uses the uncapped "dot product * intensity" value and does ^ 17

and the other ones gets the uncapped final light value (ambient + dot * intensity) and subtracts 1

Sorry for the long comment. I hope this helps the SL lights implementation in the mod.