Wrong shading

[Flamefire]

@Flow86 @MarcusSt The shading calculation at https://github.com/Return-To-The-Roots/s25client/blob/master/src/GameWorldBase.cpp#L1221 seems to be off. Shouldn't that be 1,2,3?

Going to fix that if there is no specific reason for that.

BTW: The algorithm is different then from e.g. http://settlers2.net/documentation/world-map-file-format-wldswd/

[Spikeone]

Good luck I still had this picture

[Flamefire]

Which map is this?

[Spikeone]

Island of Hills (OMAP00.SWD) in that case

[Flamefire]

Ok. Then I'll look into shadow calculation again. RTTR seems pretty flat compared to s2

[Spikeone]

Yeah, quite strange - I just know that you may check the mapeditor by xaser, he nearly had a fully working shading with SDL which looks great. When the first rttr versions appeared, everything was lighter than its now ^^

[Flamefire]

Xaser uses the original method via gouroud shading:

The editor calculates shading (light) values for each vertex. Note that the gouraud shading in settlers 2 is precalculated. The game trusts this data and uses it to get the right color value from one of the GOUx.DAT-files (x = 5-Greenland, 6-Wasteland, 7-Winterland). This files contain a color value for each of the light values the editor set. If you plan to change this block to see what happens, you have to load this map in the game to see it. The editor is not interested in this block cause it calculates the values from scratch.

If you want to know how the original settlers 2 editor and game calculates the ShadingInformation, take a look at: http://bazaar.launchpad.net/~xaser/%2Bjunk/s2mapeditor/annotate/head%3A/CMap.cpp The function called 'modifyShading(.....)' is what you need.

In the GOUx.DAT files there are color values ( = color indices = mapped rgb data 8bpp). If you want to shade a pixel, you have to read it from the tileset (the 8bit pixel value) and you need the shading value from this block (more specifically the interpolated value at the point you want to draw). The GOUx.DAT files are two-dimensional arrays, for every of the 256 possible shading values (0x00 - 0xFF) every shaded palette is saved in this array. So do the following to get the new color value for a given color value and a given shading value:

new_color_value = $GOUx.DAT[$shading_value][$color_value]

If I get this right, this means we need the shading for each point, for every pixel per triangle we need to interpolate the shading_value from these shadings and then index the GOU-array for the current landscape with the shading_value and the current pixel value to get the correct palette index we can then use to draw the pixel.
This sounds very hard and complicated to do with OGL...

[Spikeone]

http://graphics.cs.cmu.edu/nsp/course/15-462/Spring04/slides/08-shading.pdf http://stackoverflow.com/questions/14796313/cant-get-gouraud-shading-in-opengl-to-work http://foerterer.org/cpp/licht/interpolation.htm http://web.media.mit.edu/~gordonw/OpenGL/

On a first glance this doesn't sound really hard at all - I just played around a bit with OGL but... if these examples really describe gouraud shading and this would lead to a s2-like output - it would be great and worth the effort in my opinion. Flat maps are really a problem - at least from mapdesigners perspective. Also I really liked the original "look" which is still just 95% complete at all. So, Maybe that helps :)

[Flamefire]

I can't get it to work :( My try is here: https://github.com/Flamefire/s25client/tree/shading_test if anyone wants to continue that.

[Flow86]

we can't use real opengl lighting since we arent 3D

[Spikeone]

So, just a question - we use SDL and WinAPI - why can't we use Xasers' work then?

[Flamefire]

@Flow86 Well all that OpenGL needs for lighting is a normal. It doesn't really care about 3D in that case, does it? Or would it actually need the 3rd dimension for the directional light to see, what the light cannot reach?

@Spikeone Xaser used the original method: He maps each pixel via the pre-calculated Gouroud shading data to the pallette colors and draws this. This would draw a lot of performance as currently we just draw plain textures with a bit of adjusted brightness. This is done in Hardware while the original method uses "Software shading"

I actually have another idea I'm going to try today: Problem seems to be that we don't have enough brightness range. The original is as well brighter as also darker. I'm going to increase the range of supported brightness and see if this already helps. Besides that, I took a peek at the shading code of widelands. They also calculate normals (although I failed to understand their calculations and doubt that they are correct) and change the brightness according to the scalar product of the average normal and a sun vector. This could also be an option but as mentioned, the code does not look right to me. They can also have negative factors IMO which is definitely wrong. So I first try the higher range approach.

[Flow86]

@Flamefire well the directional light does not work in ortho view I think, since you dont have higher and lower places there, only one plane (so its all the same light value)

what we do is vertex shading (each point in the triangle has a different (darker) color than white, interpolating the triangle "edge" color. if these dark enough, you create the illusion of shaded hills, perhaps we simply have to adjust these values (perhaps based on the old shading tables?)

@Spikeone yea xasers method is rendering "pixel by pixel" not whole triangles at once. thats quite fast if you do it right, but the hardware shading should be better.

[Flamefire]

It actually would be good to have a reference. That is that mapping of color + shading -> color of S2. AFAIK the data flow is like: Get Pixel color-index from texture and shading (both 0-255). Then index the gouroud table: new_color_value = $GOUx.DAT[$shading_value][$color_value] (again 0-255) and use that to index into the palette. So what I would like to have is a (picture) table like this:

Color0	Shade0_0	Shade0_1	...	Shade0_255
Color1	Shade1_0	Shade0_1	...	Shade1_255
...
Color255	Shade255_0	Shade255_1	...	Shade255_255

But I try the increased range first...

[Spikeone]

Ah, I see. So it could be an option to calculate it once and only update the shading if heights are changed (by building). Also maybe not the range is a problem but the value that is added when increasing/decreasing. In S2 you were always kinda able to actually see the border - so maybe that could fix the problem, having less colors so you need to reach a certain light level and then the color gets lighter/darker.

(Just thoughts I got)

[Flamefire]

I extracted the colors from the game. At least I think that worked. Left most column are the original colors, and next to them the shades from 0-0x80 (all possible values)

Code for reference:

const int sqSize = 10;
const int w = 0x82*sqSize; const int h = 256*sqSize;
std::vector<unsigned char> buffer(w*h, libsiedler2::TRANSPARENT_INDEX);
std::vector<unsigned char> gou(256*256);
std::ifstream fGou(GetFilePath(GAMEDIR "/DATA/TEXTURES/GOU5.DAT"), std::ios_base::binary);
fGou.read(reinterpret_cast<char*>(&gou.front()), gou.size());
int x = 0; int y = 0;
for(int clr = 0; clr < 256; clr++)
{
    for(int j=0; j<sqSize; j++){
        for(int shade = -1; shade < 0x81; shade++)
        {
            int curTexIdx = (shade<0) ? clr : gou[clr+shade*256];
            for(int i=0; i<sqSize; i++){
                buffer[x+y*w] = curTexIdx;
                x++;
            }
        }
        y++;
        x=0;
    }
}
libsiedler2::ArchivItem_Palette* palette = GetTexPaletteN(1);
glArchivItem_Bitmap_Raw* bitmap = new glArchivItem_Bitmap_Raw;
bitmap->create(w, h, &buffer.front(), w, h, libsiedler2::FORMAT_PALETTED, palette);
bitmap->setPalette(palette);
bitmap->setFormat(libsiedler2::FORMAT_PALETTED);
libsiedler2::ArchivInfo archiv;
archiv.push(bitmap);

libsiedler2::Write(GAMEDIR "/DATA/texTest.bmp", archiv);

[Flamefire]

Ok so far. I found that Xaser is not using the S2 method for shading (as the GOUx.DAT is buggy) but calculates normals and a factor out of that. He basically gets a float value from that. This is multiplied with 2^16 to store it in an integer value. Then this is applied to the current color (I is the interpolated value at that point):

  r = ( (((pixel_value & dest->format->Rmask) >> dest->format->Rshift )*I) >>16 );
  g = ( (((pixel_value & dest->format->Gmask) >> dest->format->Gshift )*I) >>16 );
  b = ( (((pixel_value & dest->format->Bmask) >> dest->format->Bshift )*I) >>16 );
  r8 = (Uint8)(r > 255 ? 255 : (r < 0 ? 0 : r));
  g8 = (Uint8)(g > 255 ? 255 : (g < 0 ? 0 : g));
  b8 = (Uint8)(b > 255 ? 255 : (b < 0 ? 0 : b));

I implemented that so far and the points now have the same shading value (new integer property) as in Xasers code. I tried to set the OGL color to half the original float (color = shading / (1 << 16) / 2) because it is multiplied with 2 in OGL (required to make it brighter then original) but it is not working as expected. Everything is too bright and it seems to be the wrong way round (lighter stuff is darker):

https://github.com/Flamefire/s25client/tree/shadingTest2

[Flamefire]

I forgot to remove the normal based shading. Now it looks like that (left RRTR with this changes, right Xasers editor):

[Flamefire]

Hm. @Spikeone s Screenshot is not reproducible with current map editor as it seems. The best result I got so far is this:

[Spikeone]

Press 'p' to Emanze S2 shading ;)

[Flamefire]

Easteregg? :D Gonna check that

[Spikeone]

Nah, because it never perfectly wirkend he just added it as hotkey for better testing ;)

[Flamefire]

Ah I see.

I implemented a more S2 like shading. The normal-based shading has the problem, that it does not take shadows into account. A mountain in the direction of the light should throw a shadow over the field below. This is done in S2. Result looks like this:

[Spikeone]

I think this is alreay a good step towards better shading - compared to s2 this still seems wrong as areas with light shadows do not really appear darker. I made a comparism between s2 - flamefire shading and I guess missing darkness there is the point why maps look so flat.

[MarcusSt]

The difference is maybe in the shading model (or the number of bits) applied in S2. RttR uses glShadeModel(GL_SMOOTH), as it is the only option (GL_FLAT means exactly one color per triangle). S2 seems to interpolate shades in big steps, therefore always multiplying whole areas with the same factor. The smoother steps in RttR make it less obvious that there's a difference in shade. There are many completely inefficient ways to implement the S2 shading in RttR like dividing the triangles into sub-polygons with flat shading that represent the S2 areas or putting a "shading layer" (new layer with texture containing shade values) on top etc.

See also:

[Flamefire]

Yes that is the problem: If there is a hard border between shades then the difference appears bigger than when you have a smooth transition. In RTTR we have 24 bits for the colors so we interpolate between all of them. S2 is 8 bit only. So you see those "steps"

Also S2 has different values per color. That is not easy for RTTR. So IMO we can leave it with the one I made... At least until someone comes up with a better solution.

[MarcusSt]

Agreed. We should not spend too much time on that, as we will never reach perfection anyway.

[Flamefire]

Well, one could use 3D rendering... ;) But yes for 0.8.3 at least I'd consider this done with this commit if noone objects

[Spikeone]

Why should it be that hard to use less gray steps? Currently it doesnt really feel like anythibg changed. We can use 24 instead of 8 bit but - who should we. It should be also possible to use less colors then for shading Addons the Illusion...

[Flamefire]

Well its simply not possible or at least not reasonable. The driver is used for interpolation, and you can't influence that much.

[Flamefire]

Any objections if I merge the changes? I think it looks better than before so it can go out

[MarcusSt]

Nope, please go ahead. We won't get closer without writing our own shader. This may happen some day, but shouldn't be of any priority now.