There is something wrong with normal maps

[MatthewsSam]

I did some test of normal map on simple objects and the results are strange. When you enable normal maps it looks like part where you have the UV seams are set to hard.

Here is an example, same mesh, same parameters with/without normal map enabled

And without normal map

At first I thought there was something wrong with the mesh, but after looking in blender, the mesh is displayed without any issues

[MatthewsSam]

After some investigation, somebody at work guided me and found the algorithm used by blender.

https://github.com/tcoppex/ext-mikktspace

We should find how to plug it in stk

[Benau]

With https://gist.github.com/Benau/f8936ffa33557f554ecdda853668a1af

[Benau]

Now it's WYSIWYG, and I think bitangent can be calculated by cross (tangent, normal) * bitangent_sign (packed into 2bit of 10101012 from tangent vector), so I hope devee doesn't mind for each unqiue vertex 4 bytes more in spm file... (maybe we can find a way to export only tangents if the mesh use normalmap)

[deveee]

A lot of things may happen before 0.9.4, maybe google will even increase the apk size limit ;)

Or it's still possible to:

• implement obb files
• compress everything using lzma and decompress when extracting in the first run
• convert shared textures to jpg and then restore to png when extracting
• write small app in java that will extract additional tracks

So I'm fine with file size increase if it's needed.

[Benau]

Now @deveee and @auriamg it's ready to test Recommend to use a special materals.xml https://gist.github.com/Benau/1e745d239e2209a7b00622cd2b481e3b (you may want to move deprecated/coniferous_tree_alpha.jpg to .. for alpha mask working in snowmountain)

Wip branch: https://github.com/Benau/stk-code/tree/combine (notice: for fair test please disable global illumination and build in release), sam agreee to remove GI, you can check irc log for details

Features: uniform drawing method for gles and desktop (now shadow is possible in android)

bindless / array texture (when texture size <= 256, currently it unconditionally scale all textures in game to use 256 * 256, devee do you get good performance improve with array texture? if it is on you should see "SPTextureManager: Enable array textures, size 256" in log, if you think array textures is a good idea then later we can optimize some assets to use texture size smartly)

combined pass 1 and pass 2 solid with normal map fixed

easier shaders for sam

srgb framebuffer is false now, and for image should be srgbed, it is converted to linear (deve wants this?) (notice not for texture compression, because it seems lose some quality when srgb -> linear), and no more pow 2.2 in shaders anymore

....

Any serious regression / something else?

and devee I think your addons pack is a good idea.

[qwertychouskie]

CC @robclark

[qwertychouskie]

@Benau :

SuperCreepyKart ;P

Intel HD 4000, latest code from combine branch.

qwerty@qwerty-Inspiron-3520:~/STK/combine/stk-code/build$ bin/supertuxkart
[verbose  ] main: Error messages and other text output will be logged to /home/qwerty/.config/supertuxkart/0.8.2/stdout.log.
[info   ] [FileManager]: Data files will be fetched from: '../data/'
[info   ] [FileManager]: Data files will be fetched from: '../data/../../stk-assets/'
[info   ] [FileManager]: User directory is '/home/qwerty/.config/supertuxkart/0.8.2/'.
[info   ] [FileManager]: Addons files will be stored in '/home/qwerty/.local/share/supertuxkart/addons/'.
[info   ] [FileManager]: Screenshots will be stored in '/home/qwerty/.local/share/supertuxkart/screenshots/'.
[info   ] [FileManager]: User-defined grand prix will be stored in '/home/qwerty/.local/share/supertuxkart/grandprix/'.
[info   ] [FileManager]: Asset 0 will be loaded from '../data/challenges/'.
[info   ] [FileManager]: Asset 1 will be loaded from '../data/gfx/'.
[info   ] [FileManager]: Asset 2 will be loaded from '../data/grandprix/'.
[info   ] [FileManager]: Asset 3 will be loaded from '../data/gui/'.
[info   ] [FileManager]: Asset 4 will be loaded from '../data/../../stk-assets/library/'.
[info   ] [FileManager]: Asset 5 will be loaded from '../data/../../stk-assets/models/'.
[info   ] [FileManager]: Asset 6 will be loaded from '../data/../../stk-assets/music/'.
[info   ] [FileManager]: Asset 7 will be loaded from '../data/replay/'.
[info   ] [FileManager]: Asset 8 will be loaded from '../data/../../stk-assets/tracks/'.
[info   ] [FileManager]: Asset 9 will be loaded from '../data/../../stk-assets/sfx/'.
[info   ] [FileManager]: Asset 10 will be loaded from '../data/shaders/'.
[info   ] [FileManager]: Asset 11 will be loaded from '../data/skins/'.
[info   ] [FileManager]: Asset 12 will be loaded from '../data/../../stk-assets/textures/'.
[info   ] [FileManager]: Asset 13 will be loaded from '../data/ttf/'.
[info   ] [FileManager]: Asset 14 will be loaded from '../data/po/'.
[verbose  ] translation: Env var LANGUAGE = 'en'.
[verbose  ] translation: Language 'English'.
Irrlicht Engine version 1.8.0
Linux 4.13.0-21-generic #24-Ubuntu SMP Mon Dec 18 17:29:16 UTC 2017 x86_64
[warn   ] [IrrDriver Temp Logger]: Level 2: No sRGB framebuffer available.

[info   ] IrrDriver: OpenGL version: 4.2
[info   ] IrrDriver: OpenGL vendor: Intel Open Source Technology Center
[info   ] IrrDriver: OpenGL renderer: Mesa DRI Intel(R) Ivybridge Mobile 
[info   ] IrrDriver: OpenGL version string: 4.2 (Core Profile) Mesa 17.2.2
[info   ] GLDriver: AMD Vertex Shader Layer Present
[info   ] GLDriver: ARB Shader Viewport Layer Array Present
[info   ] GLDriver: ARB Buffer Storage Present
[info   ] GLDriver: ARB Base Instance Present
[info   ] GLDriver: ARB Draw Indirect Present
[info   ] GLDriver: ARB Compute Shader Present
[info   ] GLDriver: ARB Arrays of Arrays Present
[info   ] GLDriver: ARB Texture Storage Present
[info   ] GLDriver: ARB Texture View Present
[info   ] GLDriver: ARB Image Load Store Present
[info   ] GLDriver: ARB Shader Atomic Counters Present
[info   ] GLDriver: ARB Shader Storage Buffer Object Present
[info   ] GLDriver: ARB Multi Draw Indirect Present
[info   ] GLDriver: EXT Texture Compression S3TC Present
[info   ] GLDriver: ARB Texture Compression RGTC Present
[info   ] GLDriver: ARB Uniform Buffer Object Present
[info   ] GLDriver: ARB Explicit Attrib Location Present
[info   ] GLDriver: EXT Texture Filter Anisotropic Present
[info   ] GLDriver: Geometry Shaders Present
[info   ] GLDriver: ARB Texture Swizzle Present
[info   ] GLDriver: ARB Pixel Buffer Object Present
[info   ] GLDriver: ARB framebuffer sRGB Present
[info   ] GLDriver: ARB Sampler Objects Present
[info   ] GLDriver: ARB Vertex Type 2_10_10_10_rev Present
[warn   ] irr_driver: Created visual is not sRGB-capable. Re-creating device to workaround the issue.
[warn   ] [IrrDriver Temp Logger]: Level 2: No sRGB framebuffer available.

[info   ] IrrDriver: OpenGL version: 4.2
[info   ] IrrDriver: OpenGL vendor: Intel Open Source Technology Center
[info   ] IrrDriver: OpenGL renderer: Mesa DRI Intel(R) Ivybridge Mobile 
[info   ] IrrDriver: OpenGL version string: 4.2 (Core Profile) Mesa 17.2.2
[info   ] GLDriver: AMD Vertex Shader Layer Present
[info   ] GLDriver: ARB Shader Viewport Layer Array Present
[info   ] GLDriver: ARB Buffer Storage Present
[info   ] GLDriver: ARB Base Instance Present
[info   ] GLDriver: ARB Draw Indirect Present
[info   ] GLDriver: ARB Compute Shader Present
[info   ] GLDriver: ARB Arrays of Arrays Present
[info   ] GLDriver: ARB Texture Storage Present
[info   ] GLDriver: ARB Texture View Present
[info   ] GLDriver: ARB Image Load Store Present
[info   ] GLDriver: ARB Shader Atomic Counters Present
[info   ] GLDriver: ARB Shader Storage Buffer Object Present
[info   ] GLDriver: ARB Multi Draw Indirect Present
[info   ] GLDriver: EXT Texture Compression S3TC Present
[info   ] GLDriver: ARB Texture Compression RGTC Present
[info   ] GLDriver: ARB Uniform Buffer Object Present
[info   ] GLDriver: ARB Explicit Attrib Location Present
[info   ] GLDriver: EXT Texture Filter Anisotropic Present
[info   ] GLDriver: Geometry Shaders Present
[info   ] GLDriver: ARB Texture Swizzle Present
[info   ] GLDriver: ARB Pixel Buffer Object Present
[info   ] GLDriver: ARB framebuffer sRGB Present
[info   ] GLDriver: ARB Sampler Objects Present
[info   ] GLDriver: ARB Vertex Type 2_10_10_10_rev Present
[info   ] SharedGPUObjects: Hardware Skinning enabled, method: TBO, max bones: 1024
[info   ] ShaderFilesManager: Compiling shader : sp_pass.vert
[info   ] ShaderFilesManager: Compiling shader : sp_solid.frag
[info   ] ShaderFilesManager: Compiling shader : sp_shadow.vert
[info   ] ShaderFilesManager: Compiling shader : white.frag
[info   ] ShaderFilesManager: Compiling shader : sp_skinning.vert
[info   ] ShaderFilesManager: Compiling shader : sp_skinning_shadow.vert
[info   ] ShaderFilesManager: Compiling shader : sp_decal.frag
[info   ] ShaderFilesManager: Compiling shader : sp_alpha_test.frag
[info   ] ShaderFilesManager: Compiling shader : sp_shadow_alpha_test.frag
[info   ] ShaderFilesManager: Compiling shader : sp_unlit.frag
[info   ] ShaderFilesManager: Compiling shader : sp_normal_map.frag
[info   ] ShaderFilesManager: Compiling shader : sp_grass_pass.vert
[info   ] ShaderFilesManager: Compiling shader : sp_grass.frag
[info   ] ShaderFilesManager: Compiling shader : sp_grass_shadow.vert
[info   ] ShaderFilesManager: Compiling shader : sp_transparent.frag
[info   ] ShaderFilesManager: Compiling shader : sp_ghost.frag
[info   ] ShaderFilesManager: Compiling shader : sp_displace.frag
[info   ] ShaderFilesManager: Compiling shader : colorize.frag
[info   ] ShaderFilesManager: Compiling shader : sp_normal_visualizer.vert
[info   ] ShaderFilesManager: Compiling shader : sp_normal_visualizer.geom
[info   ] irr_driver: GLSL supported.
[info   ] ShaderFilesManager: Compiling shader : texturedquad.vert
[info   ] ShaderFilesManager: Compiling shader : texturedquad.frag
[info   ] ShaderFilesManager: Compiling shader : uniformcolortexturedquad.frag
[info   ] ShaderFilesManager: Compiling shader : colortexturedquad.vert
[info   ] ShaderFilesManager: Compiling shader : colortexturedquad.frag
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/saved-session/
[info   ] HTTPRequest: Downloading http://addons.supertuxkart.net/dl/xml/news.xml
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for 4294967295, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for 4294967295, skipping unlock
[info   ] addons: Downloading updated addons.xml.
[info   ] HTTPRequest: Downloading http://addons.supertuxkart.net/dl/xml/assets.xml
[error  ] STKTexManager: Texture name is empty.
[error  ] STKTexManager: Texture ../data/../../stk-assets/wip-tracks/20_mansion/screenshot.jpg not found or invalid.
[error  ] STKTexManager: Texture ../data/../../stk-assets/wip-tracks/20_newton/screenshot.jpg not found or invalid.
[error  ] STKTexManager: Texture ../data/../../stk-assets/wip-tracks/city_catalina/screenshot.jpg not found or invalid.
[error  ] STKTexManager: Texture ../data/../../stk-assets/wip-tracks/pbr_test/screenshot.jpg not found or invalid.
[error  ] STKTexManager: Texture /home/qwerty/.local/share/supertuxkart/addons/tracks/1738/screenshot.png not found or invalid.
[warn   ] track: Music information file 'amazon_jungle.music' not found for track 'Amazonian Journey' - ignored.

[warn   ] Track: NavMesh is not found for arena Area 498, disable AI for it.

[warn   ] Track: NavMesh is not found for arena Circle Of Destiny, disable AI for it.

[warn   ] track: Music information file 'TheDeadRocks-EasyJob.music' not found for track 'Coyote Canyon' - ignored.

[warn   ] track: Music information file 'ChillCarrier-Druckverlust.music' not found for track 'Doing the Dishes' - ignored.

[error  ] STKTexManager: Texture name is empty.
[warn   ] track: Music information file 'shootingstars.music' not found for track 'Fast kart does amazing drive' - ignored.

[warn   ] TrackManager: Track 'addon_fastkartdoesamazingdrive.old' is not supported by this binary, ignored. (Track is version 1, this executable supports from 6 to 7).
[warn   ] track: Music information file 'dance_cleopatra.music' not found for track 'Hacienda [edit]' - ignored.

[error  ] STKTexManager: Texture /home/qwerty/.local/share/supertuxkart/addons/tracks/ice_cave/screenshot.jpg not found or invalid.
[warn   ] track: Music information file 'jumping_to_the_stars.music' not found for track 'Las Dunas Stadium' - ignored.

[warn   ] track: Music information file 'TheDeadRocks-GreenSecretFromAmazonia.music' not found for track 'Lost Speedway' - ignored.

[warn   ] Track: NavMesh is not found for arena Magma Battle, disable AI for it.

[warn   ] Track: NavMesh is not found for arena Mall, disable AI for it.

[warn   ] track: Music information file 'ChillCarrier-Druckverlust.musi' not found for track 'Night by the Sea' - ignored.

[warn   ] track: Music information file 'Ethereal_Spectrum.music' not found for track 'Old Star Track' - ignored.

[error  ] STKTexManager: Texture /home/qwerty/.local/share/supertuxkart/addons/tracks/resortnew/sshot-mountain.jpg not found or invalid.
[info   ] HTTPRequest: Downloading http://addons.supertuxkart.net/dl/images/32999604857af930517f49.png
[warn   ] track: Music information file 'shootingstars.music' not found for track 'Shooting Karts' - ignored.

[warn   ] Track: NavMesh is not found for arena Soccer arena, disable AI for it.

[warn   ] Track: NavMesh is not found for arena STK Mercury, disable AI for it.

[warn   ] track: Music information file 'dance_cleopatra.music' not found for track 'The Old Island' - ignored.

[warn   ] track: Music information file 'none.music' not found for track 'whiteness' - ignored.

[error  ] STKTexManager: Texture /home/qwerty/.local/share/supertuxkart/addons/tracks/whiteness/screenshot.jpg not found or invalid.
[info   ] GrandPrixManager: Loading Grand Prix files from ../data/grandprix/
[info   ] GrandPrixManager: Loading Grand Prix files from /home/qwerty/.local/share/supertuxkart/grandprix/
[error  ] GrandPrixData: The Grand Prix file '/home/qwerty/.local/share/supertuxkart/grandprix/usr_gp_11486751.grandprix' contains a track 'city' that does not exist
[error  ] GrandPrixManager: Ignoring Grand Prix /home/qwerty/.local/share/supertuxkart/grandprix/usr_gp_11486751.grandprix (Unknown track)

[error  ] GrandPrixData: The Grand Prix file '/home/qwerty/.local/share/supertuxkart/grandprix/usr_gp_37478225.grandprix' contains a track 'city' that does not exist
[error  ] GrandPrixManager: Ignoring Grand Prix /home/qwerty/.local/share/supertuxkart/grandprix/usr_gp_37478225.grandprix (Unknown track)

[warn   ] KartModel: dat boi.spm has no striaght frame defined.
[error  ] STKTexManager: Error while loading kart 'Katie':
[error  ] STKTexManager: Failed to load genericshadow.png.
[error  ] STKTexManager: Error while loading kart 'Kiki (Cyber Squirrel)':
[error  ] STKTexManager: Failed to load genericshadow.png.
[warn   ] Material: Cannot determine texture full path: unicolor_white
[warn   ] KartModel: robbie rotten.spm has no striaght frame defined.
[warn   ] KartModel: sportaflop.spm has no striaght frame defined.
[info   ] ShaderFilesManager: Compiling shader : coloredquad.vert
[info   ] ShaderFilesManager: Compiling shader : coloredquad.frag
[info   ] Achievements: Synching achievement 1,2,3,4,5,6,7,8,9,10,11 to server.
[info   ] HTTPRequest: Sending userid=15980&token=************************&achievementid=1[info   ] HTTPRequest: Sending userid=15980&token=************************&visitingid=15980 to https://addons.supertuxkart.net/api/v2/user/get-friends-list/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] ShaderFilesManager: Compiling shader : screenquad.vert
[info   ] ShaderFilesManager: Compiling shader : IBL.frag
[info   ] ShaderFilesManager: Compiling shader : sunlight.frag
[info   ] ShaderFilesManager: Compiling shader : pointlight.vert
[info   ] ShaderFilesManager: Compiling shader : pointlight.frag
[info   ] ShaderFilesManager: Compiling shader : combine_diffuse_color.frag
[info   ] ShaderFilesManager: Compiling shader : glow.frag
[info   ] ShaderFilesManager: Compiling shader : tonemap.frag
[info   ] ShaderFilesManager: Compiling shader : passthrough.frag
[info   ] ShaderFilesManager: Compiling shader : linearizedepth.frag
[info   ] ShaderFilesManager: Compiling shader : ssao.frag
[info   ] ShaderFilesManager: Compiling shader : bilateralH.comp
[info   ] ShaderFilesManager: Compiling shader : bilateralV.comp
[info   ] ShaderFilesManager: Compiling shader : mlaa_color1.frag
[info   ] ShaderFilesManager: Compiling shader : mlaa_blend2.frag
[info   ] ShaderFilesManager: Compiling shader : mlaa_neigh3.frag
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/poll/
[info   ] HTTPRequest: Sending userid=15980&token=************************ to https://addons.supertuxkart.net/api/v2/user/client-quit/
[verbose  ] Thread: Start waiting 2221540.311000
[verbose  ] Thread: Waited 0.646000 seconds for thread to become deleteable.
[verbose  ] Thread: Stop waiting 2221540.957000
[info   ] Singleton: Destroyed singleton.
[info   ] Singleton: Destroyed singleton.
[info   ] Singleton: Destroyed singleton.

[qwertychouskie]

It only seems to happen with Texture Compression on (and I did try clearing the cache, no difference).

[deveee]

Atm. only fps comparison:

SP intel hd4000 level 6 no GI 1024x768
- hacienda 22 fps
- cornfield 19 fps
- cocoa 13 fps

SP intel hd4000 level 2 1024x768
- hacienda 240 fps
- cornfield 190 fps
- cocoa 110 fps

SP nvidia 635m level 6 no GI 1024x768
- hacienda 26 fps
- cornfield 22 fps
- cocoa 16 fps

SP nvidia 635m level 2 1024x768
- hacienda 240 fps
- cornfield 200 fps
- cocoa 130 fps

master intel hd4000 level 6 no GI 1024x768
- hacienda 19 fps
- cornfield 16 fps
- cocoa 12 fps

master intel hd4000 level 2 1024x768
- hacienda 230 fps
- cornfield 165 fps
- cocoa 93 fps

master nvidia 635m level 6 no GI 1024x768
- hacienda 25 fps
- cornfield 19 fps
- cocoa 14 fps

master nvidia 635m level 2 1024x768
- hacienda 230 fps
- cornfield 188 fps
- cocoa 120 fps

In all cases it's slightly faster in combine branch.

There is a bug on intel HD4000 and all karts are red for me. Though actually they look fine.

And autumn cocoa:

I will look if there are some regressions when playing the game. And I will test it on android too.

And yes I prefer disabled srgb framebuffer, because we need it for android anyway... And it would make GL and GLES renderers more similar. But last time I couldn't make transparent objects 100% correct. And I'm not sure if anti aliasing is 100% correct (though it looks fine for me).

[deveee]

On intel hd4000 texture array is slightly slower. Max texture size 512 gives 21 fps and 256 gives 18 fps.

And compiler complains:

./../src/graphics/draw_calls.cpp: In member function 'void DrawCalls::parseSceneManager(irr::core::list<irr::scene::ISceneNode*>&, const irr::scene::ICameraSceneNode*)':
./../src/graphics/draw_calls.cpp:148:35: error: use of 'node' before deduction of 'auto'
                 for (auto* node : node->getAllNodes())

for (auto* n : node->getAllNodes()) should work?

[deveee]

It was also complaining about missing GL_COMPRESSED_RGBA_S3TC_DXT5_EXT, but it's just one more #ifndef USE_GLES2.

And on intel some textures are constantly moving. abyss_2017.12.24_00.08.27.webm.zip

I didn't notice any other issues after quick test.

And I will look how it works on android tomorrow.

[deveee]

Actually I found some time.

On android texture array is a bit slower too. For example in abyss 18fps when enabled, 20fps when disabled.

Performance is similar:

SP (disabled texture array):

• hacienda 32 fps
• cornfield 30 fps
• abyss 20 fps

0.9.3

• hacienda 33 fps
• cornfield 31 fps
• abyss 22 fps

Shadows indeed work, it's not realtime rendering though ;)

And I had some crashes (when changing to advanced pipeline and when entering kart selection screen), but I compiled it in release mode, so I don't have the backtraces.

And it looks that everything renders properly.

[Benau]

Can you try in windows if there is red kart for hd4000? ... i hope i don't need fix mesa for wrong 10 bit vector decompress....

[Benau]

And if array texture give no performance boost, i will just define them out for final commit...

[Benau]

Actually sometime ago you told me ubo for android is a little slower, can you try in sp_base.cpp:1479 do the commented out code

    void* ptr = glMapBufferRange(GL_UNIFORM_BUFFER, 0,
        (16 * 9 + 2) * sizeof(float), GL_MAP_WRITE_BIT |
        GL_MAP_UNSYNCHRONIZED_BIT | GL_MAP_INVALIDATE_BUFFER_BIT);
    memcpy(ptr, g_stk_sbr->getShadowMatrices()->getMatricesData(),
        (16 * 9 + 2) * sizeof(float));
    glUnmapBuffer(GL_UNIFORM_BUFFER);

instead of glbuffersubdata? (this should be safe, because there are 3 ubo buffer-interchange, so you should never write to a buffer currently used by opengl)

If not than maybe gles doesn't like instance rendering ... though I would want to only 1 draw method ... sorry...

btw now we only have 1 set of shader, i think we can add some shading to meshes for advanced pipeline off more easier, by mixing the normal from vertex shader (want?)

[Benau]

In mesa intel-related source it says

 Prior to Haswell, the hardware can't natively support GL_FIXED or
 2_10_10_10_REV vertex formats.  This pass inserts extra shader code
 to produce the correct values.

in mesa/src/intel/compiler/brw_nir_attribute_workarounds.c function apply_attr_wa_block:

      if (wa_flags & BRW_ATTRIB_WA_NORMALIZE) {
         /* ES 3.0 has different rules for converting signed normalized
          * fixed-point numbers than desktop GL.
          */
         if ((wa_flags & BRW_ATTRIB_WA_SIGN) &&
             !state->use_legacy_snorm_formula) {
            /* According to equation 2.2 of the ES 3.0 specification,
             * signed normalization conversion is done by:
             *
             * f = c / (2^(b-1)-1)
             */
            nir_ssa_def *es3_normalize_factor =
               nir_imm_vec4(b, 1.0f / ((1 << 9) - 1), 1.0f / ((1 << 9) - 1),
                               1.0f / ((1 << 9) - 1), 1.0f / ((1 << 1) - 1));
            val = nir_fmax(b,
                           nir_fmul(b, nir_i2f32(b, val), es3_normalize_factor),
                           nir_imm_float(b, -1.0f));
         } else {
            /* The following equations are from the OpenGL 3.2 specification:
             *
             * 2.1 unsigned normalization
             * f = c/(2^n-1)
             *
             * 2.2 signed normalization
             * f = (2c+1)/(2^n-1)
             *
             * Both of these share a common divisor, which we handle by
             * multiplying by 1 / (2^b - 1) for b = <10, 10, 10, 2>.
             */
            nir_ssa_def *normalize_factor =
               nir_imm_vec4(b, 1.0f / ((1 << 10) - 1), 1.0f / ((1 << 10) - 1),
                               1.0f / ((1 << 10) - 1), 1.0f / ((1 << 2)  - 1));

            if (wa_flags & BRW_ATTRIB_WA_SIGN) {
               /* For signed normalization, the numerator is 2c+1. */
               nir_ssa_def *two = nir_imm_float(b, 2.0f);
               nir_ssa_def *one = nir_imm_float(b, 1.0f);
               val = nir_fadd(b, nir_fmul(b, nir_i2f32(b, val), two), one);
            } else {
               /* For unsigned normalization, the numerator is just c. */
               val = nir_u2f32(b, val);
            }
            val = nir_fmul(b, val, normalize_factor);
         }
      }

I think they should use the gles3 formula anyways for opengl >= 3.3, at least I have tested hd4600 and amd radeon in windows, the 10bit vector is corrected processed (no red karts/ correct texture matrix)

and state->use_legacy_snorm_formula is determined by !_mesa_is_gles3(&brw->ctx) in brw_vs.c For my intel hd3000 in linux when compiler with gles2 on, no more red karts, deveee do you see the same?

Do you prefer to file mesa a bug report or fix it in our shaders?

And some details: for texture matrix in sp, I use the i_normal.w as a mask(default 0), in void SPMeshBuffer::enableTextureMatrix(unsigned mat_id), i pass a 1 << 30 (1.0f when corrected normalized) to enable texture matrix for some vertex if there is animated texture for such material and in shaders it will do:

    uv = vec2(i_uv.x + (i_misc_data.x * i_normal.w),
        i_uv.y + (i_misc_data.y * i_normal.w));

[deveee]

No difference in performance when I changed glbuffersubdata to void* ptr = glMapBufferRange (...)

Indeed GLES 3.0 works fine with intel. And it works fine on windows too.

I don't know if/when they would fix it in mesa and older drivers versions will be broken anyway, so that we probably should fix it on our side.

[Benau]

btw do you need me write a etc2 texture compressor for android (like the current libsquish way)

now for all platform textures are loaded by threads, and i think we can cache it after first compression

though i don't have a phone which support etc2, you have 1?

[deveee]

ETC2 should be available in GLES 3.0. But from what I read, compression may take quite a lot of time. But I didn't make any tests.

[qwertychouskie]

@Benau :

SuperSymbolicKart ;P

Track: https://forum.freegamedev.net/viewtopic.php?f=18&t=7739 though all textures are affected, but only with texture compression.

[Benau]

devee from https://github.com/Benau/stk-code/commit/2f1a82ffaf58039ec92951a47070a659dd350898 is this true if no glenable(gl_framebuffer_srgb), than above can be removed?

and i think latest commit should work for you (ivybridge?)

also can you reproduce qwerty's black texture compression issue?

[deveee]

Yes, can be removed. And latest commit works for me on hd4000 with mesa 17.x.

Actually it's still broken with mesa 10.3, but this version is quite old, so I'm not sure if it matters. Here is how it looks:

And I can't reproduce qwerty's issue. I used clean HOME directory, so textures were not cached before. Maybe it matters?

Do you think that card contains="Ivybridge" will fix it on celeron intel HD too? Tbh. I didn't check if/how it can be recognized on linux. Maybe we should recognize intel generations based on information about CPU.

[Benau]

probably bitshift for int doesn't work? (try the previous commit c2e51f96102f22a76397d2a6a8fe604ccf85ba1d?)

No texture cache is used for SP, all compressed on-the-fly by threads, unless qwerty doesn't build the bundled libsquish... let me think if can get some more ideas

I use the names from mesa include/pci_ids/i965_pci_ids.h, not sure if you think it's safe...

[qwertychouskie]

Actually my issues are fixed with a fresh checkout of stk-assets (rather than a symlink to /usr/share/supertuxkart/data), it seems STK was picking up the wrong restrictions file. Performance on level 4 + high shadows seems a tiny bit better. The only issue I noticed is that the kart selection screen takes a bit to load.

[deveee]

c2e51f96102f22a76397d2a6a8fe604ccf85ba1d causes a lot of shaders compilation errors (on current mesa too) and then a segfault. For latest commit in combine branch d526aeb0e4a46070f0f677e59ea464a10865bb9a there is no shader compilation error (but on mesa 10.3 it looks incorrectly).

And when you changed params.HandleSRGB = false; it doesn't start on my nvidia anymore. Maybe we should set it to GLX_DONT_CARE when we request HandleSRGB = false.

[Benau]

well i would avoid to bloat stk for some old drivers, for example sp_instance use a fixed size of 32, if i use some other format the vector push_back will need to take stride into account

your thoughts?

[Benau]

https://lists.freedesktop.org/archives/mesa-dev/2017-December/180687.html

[Benau]

I make the nm visualizer more professional, now for model (like hexley) with tangent exported, you can see its tangent / bitangent in debug view

and deve maybe we can pass GL_FALSE for glVertexAttribPointer 2_10_10_10 and * a normalized factor vec4 in shader for broken mesa version ourselves.

[deveee]

I checked and mesa 11.0.0 is buggy for intel hd4000 (looks like rtt is not cleared before rendering the kart in kart selection screen and also shadows don't work properly, so that everything is dark).

But on mesa 11.2.0 everything works fine. So if we need graphical restrictions, it should be probably for mesa <11.2.

Actually Ubuntu 16.04 has mesa 11.2.0 and Debian Jessie has 13.0.6 in backports, so it's not thaat bad.

Anyway do you think that other buggy drivers will work? Maybe someone can test it with skylake on windows :P

[Benau]

876a8684003e1b9b7a49df68ff8c8c3202c721b3 should work for you even in 10.3?

And for your question about "EGL" in stk, maybe one day if i'm smart to learn vulkan and we can make stk egl + wayland + vk port (and only enable vk / egl in wayland)

[deveee]

Now the bug with messed kart models is fixed. But these two bugs (kart selection screen and shadows) persist.

Not sure if you want to fix it or just if mesa < 11.2 then disable shadows?

For EGL on linux nvidia >= 355 seems the main restriction.

[Benau]

So current master is working if mesa < 11.2?

[deveee]

Shadows don't work, there is graphical restriction:

  <card contains="Intel"          os="linux"    version="<11.2"    disable="GeometryShader"/>

and it looks exactly the same when I remove this restriction.

But kart selection screen works fine in master. I can try to find a reason if you can't reproduce it.

[Benau]

Do u see GL_AMD_vertex_shader_layer usable in ternimal try to make the vs string in CVS empty, and stk will render each shadow layer indivdually (like in gles)

Would be good if u can help for the other issue...

[deveee]

Disable vs helped for shadows.

[Benau]

actually maybe we can disable vs layer for all cards anyway (and do the same like in gles), do you see regression?

[deveee]

For broken kart selection screen it looks that glClear framebuffer doesn't work when there are more textures.

It works when I manually clear it at the beginning of drawSceneDeferred:

    unsigned char* empty[512 * 512] = {};

    glBindTexture(GL_TEXTURE_2D, m_rtts->getFBO(FBO_SP).getRTT()[0]);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 512, 512, GL_RGBA,
        GL_UNSIGNED_BYTE, empty);
    glBindTexture(GL_TEXTURE_2D, m_rtts->getFBO(FBO_SP).getRTT()[1]);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 512, 512, GL_RGBA,
        GL_UNSIGNED_BYTE, empty);
    glBindTexture(GL_TEXTURE_2D, m_rtts->getFBO(FBO_SP).getRTT()[2]);
    glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 512, 512, GL_RGBA,
        GL_UNSIGNED_BYTE, empty);
    glBindTexture(GL_TEXTURE_2D, 0);

And no, I don't see regression with disabled vs. It works fine on both intel and nvidia.

Actually current git has one more issue, but it's just for debugging:

[info   ] ShaderFilesManager: Compiling shader : sp_normal_visualizer.frag
[error  ] SPShader: Error when linking shader sp_normal_visualizer in pass 0
[error  ] SPShader: Link info
---------
error: interpolation modifier mismatch for varying parameter (named o_color) between shader stages

Mesa prints something a bit more readable

[info   ] ShaderFilesManager: Compiling shader : sp_normal_visualizer.frag
[error  ] SPShader: Error when linking shader sp_normal_visualizer in pass 0
[error  ] SPShader: error: geometry shader output `o_color' specifies flat interpolation qualifier, but fragment shader input specifies no interpolation qualifier

[Benau]

110915b7792987005d1500fa46c12329eca84081 works for you?

(or we can use glClearTexImage if it's avalible) mesa features.doc said:

  GL_ARB_clear_texture                                  DONE (i965, nv50, r600, llvmpipe, softpipe, swr)

[deveee]

Kart selection screen works in current git.

And btw. kart shadow is ok too.

[Benau]

samuncle your pbr formula is causing artifacts in stk enterprise (lighting)

Only the first revision (original formula) of combine_diffuse_color.frag works fine

[Benau]

Merged, sam said he will fix stk enterprise textures

and he will re-do the overworld, so please use the fallback overworld for now

and displace shader in non-advanced pipeline is wip

[deveee]

I finally found some time to test it on windows. And it's actually even faster than on linux. For example for intel HD4000 on highest settings and 1024x768 I have 23fps in cornfield crossing (current git, so it's not 100% the same as previous tests).

And it works just fine with texture compression. And it looks that everything is renered properly. I tested it with current drivers with OpenGL 4.0.

I had a crash though (may be the same as #3098, but I don't have a backtrace). It happened when I disabled advanced lighting and then opened kart selection screen quickly and moved mouse on different kart. It looks like some textures weren't loaded on time, or maybe something is not thread-safe. No idea... Maybe I will try it with valgrind +drd/helgrind, but it works like a tortoise.

[deveee]

Ok, I have a backtrace:

Thread 1 received signal SIGSEGV, Segmentation fault.
0x7457ca87 in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
(gdb) bt
#0  0x7457ca87 in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#1  0x7457e98e in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#2  0x7457c59b in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#3  0x7473f314 in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#4  0x7473f451 in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#5  0x7473f53a in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#6  0x74740f40 in ig7icd32!RegisterProcTableCallback ()
   from C:\Windows\SysWOW64\ig7icd32.dll
#7  0x00a14a8a in SP::SPMeshBuffer::draw (this=0x1f334a60,
    dct=SP::DCT_NORMAL, material_id=2)
    at C:/Users/Ja/Desktop/supertuxkart/stk-code/src/graphics/sp/sp_mesh_buffer.
hpp:127
#8  0x0047fa13 in SP::draw (rp=SP::RP_1ST, dct=SP::DCT_NORMAL)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\graphics\sp\sp_base.cpp:131
4
#9  0x0046d1fb in ShaderBasedRenderer::renderScene (this=0x16a52450,
    camnode=0x1f6887c8, dt=0.0170000009, hasShadow=false, forceRTT=true)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\graphics\shader_based_rend-
--Type <return> to continue, or q <return> to quit---
erer.cpp:458
#10 0x0046e944 in ShaderBasedRenderer::renderToTexture (this=0x16a52450,
    render_target=0x1f6814d8, camera=0x1f6887c8, dt=0.0170000009)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\graphics\shader_based_rende
rer.cpp:872
#11 0x0046925b in GL3RenderTarget::renderToTexture (this=0x1f6814d8,
    camera=0x1f6887c8, dt=0.0170000009)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\graphics\render_target.cpp:
130
#12 0x004de1dc in GUIEngine::ModelViewWidget::update (this=0x1f681150,
    delta=0.0170000009)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\guiengine\widgets\model_vie
w_widget.cpp:199
#13 0x004a9ae0 in GUIEngine::update (dt=0.0170000009)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\guiengine\engine.cpp:842
#14 0x0057984e in MainLoop::run (this=0x1b6b04c0)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\main_loop.cpp:266
#15 0x005779f7 in main (argc=1, argv=0x168a2b20)
    at C:\Users\Ja\Desktop\supertuxkart\stk-code\src\main.cpp:1801
(gdb)

And I tested with slightly older drivers (10.18.10.4425, GL 4.0, released in 2016.05) and it's ok too.