Add Color Detection for Background Color

[CalSeedy]

I think the modal colour returned by GetModalColour() is being ignored when checking pixels' viability.

I implemented a static method for converting the bitmap to grayscale (this also effectively divides the computation by 3, since the rgb values become the same), the algorithm still prefers White.

// Can just hardcode these values later, makes sense here to understand where it is coming from
private static readonly float[] LuminanceVector = { 0.3086f, 0.6094f, 0.0820f };
private static readonly ColorMatrix GrayscaleColourMatrix = new ColorMatrix( new float[][]{ 
        new float[] { LuminanceVector[0], LuminanceVector[0], LuminanceVector[0], 0.0f, 0.0f },
        new float[] { LuminanceVector[1], LuminanceVector[1], LuminanceVector[1], 0.0f, 0.0f },
        new float[] { LuminanceVector[2], LuminanceVector[2], LuminanceVector[2], 0.0f, 0.0f },
        new float[] { 0.0f, 0.0f, 0.0f, 1.0f, 0.0f },
        new float[] { 0.0f, 0.0f, 0.0f, 0.0f, 1.0f }
    });
// Conversion (using saturation) taken from here http://www.graficaobscura.com/matrix/index.html
public static Bitmap ConvertToGrayscale(Bitmap original)
{
    /// <summary>
    /// Convert input bitmap colour data to grayscale. Copies original's data and modifies that.
    /// </summary>
    /// <param name="original">Input (coloured) bitmap image</param>
    /// <returns>Bitmap</returns>
    Bitmap outBMP  = new Bitmap(original.Width, original.Height);

     // Easier/faster to "overwrite" a new bmp with the original image, passing in a colour matrix to use - a grayscale one
     // Using 'using' to have automatic cleanup after use
     using (Graphics g = Graphics.FromImage(outBMP))
     {
          using (ImageAttributes attrs = new ImageAttributes())
          {
              attrs.SetColorMatrix(GrayscaleColourMatrix);

              g.DrawImage(original, new Rectangle(0, 0, original.Width, original.Height), 
                        0, 0, original.Width, original.Height, GraphicsUnit.Pixel, attrs);
           }
      }

     return outBMP;
}

Test3.bmp using the TopLeftOptimiser, the grayscale converter before the WhiteSpaceFinder is initalised, with the correct settings (autoDetectBackgroundColour = true and branching based on that @ L38 in SearchMatrix.cs).

Visualising the mask, we can see the regions the mask picks as good (green) and bad (red)

It just completely ignores the colour, even after calculating a modal colour that isn't white.

[CalSeedy]

I've made decent progress with this!

Stepping through the mask filtering, I found that the key variable is the filterVal in

private int CalculateRowSum(int stampwidth, Rectangle WorkArea, int y, byte[] buffer, int depth, int width, WhitespacerfinderSettings Settings, GetBits colorEvaluation)

int filterVal = Settings.CutOffVal - Settings.Brightness;

This is one of the main limiting factors in determining viable pixels. Since the only variable in this is Settings.Brightness, I used the GetModalColor() method to calculate the sum of the modal colour's components and use that as the brightness.

private Color GetModalColor()
{
    ...
    int foo = RoundCol.GroupBy(item => item).OrderByDescending(g => g.Count()).Select(g => g.Key).First();
    Color c = Color.FromArgb(foo);
    Settings.Brightness = c.R + c.G + c.B;
    return c;
}

This implementation works for the previous (white) backgrounds, and works relatively well for the coloured ones, with the huge caveat that the mask still sees white as a viable pixel. Maybe we would add a range to this line for the filter instead of the greater than comparison?

val = (colorEvaluation.Invoke(buffer, x, y, width, depth) > filterVal) ? (byte)1 : (byte)0;

Image dump incoming: Test1

1. TopCentreOptimiser

Mask	Output

2. BottomRightOptimiser

Mask	Output

Test3

1. BottomCentreOptimiser

Mask	Output

2. TopRightOptimiser

Mask	Output

So, from the above, we see that the background is correctly identified but any colour of a lighter shade (white) will also be included in the mask. All Optimisers used with Test3.bmp produce similar masks (and results) as Test1's and Test2.bmp's - with the difference being Test3's results are the same as Test2's, just rotated. TopRight for 3 == BottomRight for 2.

[Ildesigns]

If replace filterVal with

filtervalHigh = math.min(byte.maxvalue *3, modalcolor+ (int)settings.brightness/2)

FiltervalLow = filtervalHigh - settings.brightness

And use that as a range with your suggested approach. Looks good.

[CalSeedy]

I think it may also be worth comparing the other components in the RGB/HSL values. Since the final product will have artefacts instead of a solid colour, we should maybe consider ranges on Hue (i.e. +/- 5 deg), Saturation (+/- 5% or a fixed value), and Luminance (+/- 5% or fixed value).

Further tests should include similar colours on the background, and/or very distinct colours. My reasoning is that, without the above, colours that have a similar "brightness" (not true brightness in this case, just R+G+B) will also be considered viable.

[Ildesigns]

Messing around with an idea, (All RGB) Not sure if works yet?

1) Get the most modal Color value

2) Find all the colors where

coarseMask =0xF4F4F4 //exact value tbd

coarsemark & color == coarsemark & modalcolor

3) This finds the pixels which are broadly the same color

4) Sum the RGB values for the above colors 5) Apply the filters above for a finer refinement of the colors i.e. filterlow < sumcolor <filterhigh

6) find the mean of the remaining colors so these finds the average of the filtered colors This is the background color

Code below not even run yet, and could change order of code to improve performance.

long GetbitvalColorlong(byte[] buffer, int offset) { //pads a long most significant int(32), is a sum of the colors //least significant int(32) is the color as int(32) long a = (buffer[offset + 0] + buffer[offset + 1] + buffer[offset + 2]) << 32 | //sums GetbitvalColor(buffer, offset); return a; }

    int GetbitvalColor(byte[] buffer, int offset)
    {
        //gets a color as an int (alpha stripped)
        int a = //sums
            (buffer[offset + 0] ) << 16 | //red
            (buffer[offset + 1] ) << 8 | //green
            (buffer[offset + 2] );//blue
        return a;
    }

private Color GetModalColor() { const long sumMask = 0x0FFF00000000; const long colorMask = 0xFFFFFF; const long coarseFilterMask = 0xF4F4F4; int depth; byte[] buffer; GetBitmapData(out depth, out buffer); int len = buffer.Length / depth; long[] RoundCol = new long[len];

        Parallel.For(0,len, (i) => { 
            //todo: write new function below does not work.
             RoundCol[i]=GetbitvalColor(buffer, i*depth);

        });

        long scaledOffset = Settings.Brightness << 32; //bit shift offset 32 bytes to left 
         IEnumerable<IGrouping<long,long>> colorGroups = RoundCol.GroupBy(d => d & colorMask); //group based on color as int
        long modalColor = colorGroups.Max(x => x.Count()); //most occouring Color
        long lowcolRange = (modalColor & sumMask) - scaledOffset; //cutoff filter (fine based on sum of components)
        long highColRange = (modalColor & sumMask) + scaledOffset;//cutoff filter (fine based on sum of components)

        //the below filters colors which have close sum of RGBs to the modal color (could be a completly diff color but very close sum)

        IEnumerable<long> colorGroupsRefined = colorGroups.Where(g => (g.Key & sumMask) > highColRange && (g.Key & lowcolRange) < lowcolRange).Select(h=> h.First());

        //the below filters colors which have close RGBs i.e. only similar colors.

        IEnumerable<long> cols = colorGroupsRefined.Where(x => {
            long coly = (coarseFilterMask & x);
           return coly == (coarseFilterMask & modalColor);
        });

        int meanCol = (int)cols.Select(x=>x&colorMask).Average();

        Color modalCol = Color.FromArgb(meanCol);
        return modalCol;
    }

[Ildesigns]

Ps thers definite bugs in that code like if the modal color is white (oxFFFFFF) setting the upper bound filter oxFFFFFF + brightness will have weird results!!!

More thoughts on an approach..

[CalSeedy]

The code above definitely needs more development, I spent the past couple days trying to find a way to make it work with what is there without any major success.

The result always throws an exception based on the final filter (IEnumerable<long> cols = ...) not finding any matches, and playing around with the mask values didnt help. I also tried changing the filter comparisons (> to < and visa versa) in the IEnumerable<long> colorGroupsRefined filter, and even changing the longs to ulongs (because long lowColRange tended to be a very high negative value) and trying to use that in the comparison. Still nada.

I'm unsure about next steps to try with this.

[Ildesigns]

I think I have a working version, though it isn't perfect. It works well on the block colours, can't quite crack the filtering width though. Can test on some real data this week.

Got distracted a bit messing around with Structures in place of the longs, using structlayout.explicit attribute, which is an interesting prospect of mapping multiple variables on top of each other.

[Ildesigns]

I've checked in a newe branch of my color detection code. I think we've attacked some problems in a common manner. signed -> unsigned numbers. Increase the brightness setting (about 30 gives good results for me.

Added Test 5, which isn;t working well. Though my previous test 5 without the scribble worked a lot better. I'd like to run this through some real data though. This test may be a bit OTT

[Ildesigns]

bug fix on GetbitvalColor on bit shift & color inversion