search

blitz-research / monkey2

zlib License
132 stars 44 forks source link

Any way to set GLSL uniform arrays? #468

Open DruggedBunny opened 5 years ago

DruggedBunny commented 5 years ago

Hi Mark,

I'm attempting to shove a bunch of palette values from Monkey2 side into a uniform array, but as far as I can tell, that's not currently possible as we don't have the relevant SetXXXArray option. (Did consider abusing SetMat4fArray, but... )

Naive attempt with floats! Would be looking to fill in vec3s or maybe just floats (hsl hue values) in my case...

    ' OK, this wasn't gonna happen!

    TargetImage.Material.SetFloat   ("m_Test[0]", 0.0)
    TargetImage.Material.SetFloat   ("m_Test[1]", 1.0)

    ' GLSL side:

    uniform float m_Test [2];

    if (m_Test[1] > 0.0)

    ' Nope!

I'm avoiding sending the palette texture as I would need to do up to 8, 16... maybe even 256 texture lookups per-pixel for closest-colour selection, depending on fake retro-palette, and one thing I've learned is that this is slow, whereas a quick iteration through float or vec3 arrays ought to be fine.

blitz-research commented 5 years ago

SetMat4fArray is currently your only option for sending arrays. You could store 4 colors in each element by setting the i,j,k,t values in each matrix element, or even just use the 'i' element or something to simplify lookup. I did just have a quick look at adding support for other array types and...it's complicated. It's kind of specialized for bones right now and is a bit messy, will hopefully get around to cleaning it up soon.

I'm avoiding sending the palette texture as I would need to do up to 8, 16... maybe even 256 texture lookups per-pixel for closest-colour selection

I don't really understand this. If you want a simple palette lookup, this should work:

const float NumColors=64.0; uniform sampler2D m_ColorPalette; //Should be NumColors wide, and mipmapping should be disabled for the texture.

// index in range [0,NumColors] vec4 LookUpColor(float index){ return texture2D( m_ColorPalette,vec2(index/NumColors,0.0) ); }

But I suspect this isn't quite what you want, could you describe what you're trying to do a bit more clearly?

On Wed, Feb 6, 2019 at 3:02 PM DruggedBunny notifications@github.com wrote:

Hi Mark,

I'm attempting to shove a bunch of palette values from Monkey2 side into a uniform array, but as far as I can tell, that's not currently possible as we don't have the relevant SetXXXArray option. (Did consider abusing SetMat4fArray, but... )

Naive attempt with floats! Would be looking to fill in vec3s or maybe just floats (hsl hue values) in my case...

' OK, this wasn't gonna happen!

TargetImage.Material.SetFloat ("m_Test[0]", 0.0) TargetImage.Material.SetFloat ("m_Test[1]", 1.0)

' GLSL side:

uniform float m_Test [2];

if (m_Test[1] > 0.0)

' Nope!

I'm avoiding sending the palette texture as I would need to do up to 8, 16... maybe even 256 texture lookups per-pixel for closest-colour selection, depending on fake retro-palette, and one thing I've learned is that this is slow, whereas a quick iteration through float or vec3 arrays ought to be fine.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/blitz-research/monkey2/issues/468, or mute the thread https://github.com/notifications/unsubscribe-auth/ADU3QmIlz3B92yOsd1IKV38ioaaLEpsuks5vKjfAgaJpZM4akbPg .

blitz-research commented 5 years ago

Oops, filtering should also be disabled for palette textures.

On Wed, Feb 6, 2019 at 4:47 PM Mark Sibly blitzmunter@gmail.com wrote:

SetMat4fArray is currently your only option for sending arrays. You could store 4 colors in each element by setting the i,j,k,t values in each matrix element, or even just use the 'i' element or something to simplify lookup. I did just have a quick look at adding support for other array types and...it's complicated. It's kind of specialized for bones right now and is a bit messy, will hopefully get around to cleaning it up soon.

I'm avoiding sending the palette texture as I would need to do up to 8, 16... maybe even 256 texture lookups per-pixel for closest-colour selection

I don't really understand this. If you want a simple palette lookup, this should work:

const float NumColors=64.0; uniform sampler2D m_ColorPalette; //Should be NumColors wide, and mipmapping should be disabled for the texture.

// index in range [0,NumColors] vec4 LookUpColor(float index){ return texture2D( m_ColorPalette,vec2(index/NumColors,0.0) ); }

But I suspect this isn't quite what you want, could you describe what you're trying to do a bit more clearly?

On Wed, Feb 6, 2019 at 3:02 PM DruggedBunny notifications@github.com wrote:

Hi Mark,

I'm attempting to shove a bunch of palette values from Monkey2 side into a uniform array, but as far as I can tell, that's not currently possible as we don't have the relevant SetXXXArray option. (Did consider abusing SetMat4fArray, but... )

Naive attempt with floats! Would be looking to fill in vec3s or maybe just floats (hsl hue values) in my case...

' OK, this wasn't gonna happen!

TargetImage.Material.SetFloat ("m_Test[0]", 0.0) TargetImage.Material.SetFloat ("m_Test[1]", 1.0)

' GLSL side:

uniform float m_Test [2];

if (m_Test[1] > 0.0)

' Nope!

I'm avoiding sending the palette texture as I would need to do up to 8, 16... maybe even 256 texture lookups per-pixel for closest-colour selection, depending on fake retro-palette, and one thing I've learned is that this is slow, whereas a quick iteration through float or vec3 arrays ought to be fine.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/blitz-research/monkey2/issues/468, or mute the thread https://github.com/notifications/unsubscribe-auth/ADU3QmIlz3B92yOsd1IKV38ioaaLEpsuks5vKjfAgaJpZM4akbPg .

DruggedBunny commented 5 years ago

Yeah, it's arbitrary conversion of rendered scenes to fixed palettes, so I need to scan through the available palette colours and compare 'distance'. I got a hard-coded version working (ie. m_PaletteColor0... m_PaletteColor15), but of course the code is unwieldy!

Quickly hacked out (off to work)... so the texture lookup has to be done for each pixel against each palette entry to see which fits best. I'm operating on retro-sized images so it's fine -- and still fine on 1080p, but it's a bugger to scroll through/tweak!

uniform vec3 m_Palette0;
uniform vec3 m_Palette1;
uniform vec3 m_Palette2;
uniform vec3 m_Palette3;
uniform vec3 m_Palette4;
uniform vec3 m_Palette5;
uniform vec3 m_Palette6;
uniform vec3 m_Palette7;
uniform vec3 m_Palette8;
uniform vec3 m_Palette9;
uniform vec3 m_Palette10;
uniform vec3 m_Palette11;
uniform vec3 m_Palette12;
uniform vec3 m_Palette13;
uniform vec3 m_Palette14;
uniform vec3 m_Palette15;

...

vec3 rgb_dist_check (vec3 in_color)
{
    vec3 palette_color;

    float dist = 3.402823466e+38; // Largest float
    float read_dist = dist;

    read_dist = distance (in_color, m_Palette0);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette0;
    }

    read_dist = distance (in_color, m_Palette1);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette1;
    }

    read_dist = distance (in_color, m_Palette2);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette2;
    }

    read_dist = distance (in_color, m_Palette3);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette3;
    }

    read_dist = distance (in_color, m_Palette4);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette4;
    }

    read_dist = distance (in_color, m_Palette5);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette5;
    }

    read_dist = distance (in_color, m_Palette6);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette6;
    }

    read_dist = distance (in_color, m_Palette7);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette7;
    }

    read_dist = distance (in_color, m_Palette8);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette8;
    }

    read_dist = distance (in_color, m_Palette9);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette9;
    }

    read_dist = distance (in_color, m_Palette10);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette10;
    }

    read_dist = distance (in_color, m_Palette11);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette11;
    }

    read_dist = distance (in_color, m_Palette12);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette12;
    }

    read_dist = distance (in_color, m_Palette13);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette13;
    }

    read_dist = distance (in_color, m_Palette14);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette14;
    }

    read_dist = distance (in_color, m_Palette15);

    if (read_dist < dist)
    {
        dist = read_dist;
        palette_color = m_Palette15;
    }

    return palette_color;
}

... and on MX2 side:

        Local palette_file:String = "spectrumfx2"

        Local palette:Pixmap = Pixmap.Load ("asset::shaders/" + palette_file + ".png")

        #If Not __WEB_TARGET__ ' https://github.com/blitz-research/monkey2/issues/460
            If Not palette Then Notify ("RetroFX setup error!", "You need to copy ~q" + palette_file + ".png~q from ~qretrofx/" + StripExt (palette_file) + "/~q into ~qassets/shaders/~q!", True)
        #Endif

        For Local p:Int = 0 Until palette.Width ' hard-coded 16 in glsl at present, awaiting mx2 uniform array setting!

            Local c:Color = palette.GetPixel (p, 0)

            TargetImage.Material.SetVec3f ("Palette" + p, New Vec3f (c.R, c.G, c.B))

        Next

Needs dithering, but I'm starting from scratch as I couldn't get GitHib/other dithering code to work. Also got a C64 palette and a, er, Loveless palette working.

spec

Yup, I've sent textures over successfully before, and disabled filtering! Might have another go, see how it is, but I think going for like 256 entries per pixel is gonna be bad on web, for instance...

Thanks for having a look.

DruggedBunny commented 5 years ago

Hi Mark,

I worked around this after getting some cool/nasty palette dithering code working, as posted here and hopefully testable here...

One thing that lets it down on the web is (what I assume to be) the webgl shader compilation time on hitting R for Retro, which can easily take a lengthy 15-20 seconds or more!

Once in Retro mode, [ and ] then cycle through modes.

I haven't yet figured out how (or, in fairness, tried) to get this to trigger during startup, which would be preferable.

I suppose I might need to try and render in retro mode once in OnCreateWindow without displaying the render target image, while displaying a 'building shaders' message... does that sound like the right approach, or am I missing a trick?