Issue with function BitmapToIplImage in ocv.utils

[HuguesDug]

Hello,

Now code is converted to use you wrapper and I started to loop images for treatments ;-)

My code uses DsPack (DirectX frame grabber) to grab frames from an IpCamera. The sample grabber is returning a Tbitmap. These functions run via API calls to Directx to built bitmap.

If I use this bitmap with BitmapToIplImage, it randomly leads to memory leaks or a "black" IplImage. If I display the original Bitmap in a TPicture, no problem, the bitmap is OK.

I then used another BitmatToIpl conversion routine I found on the web. No problem, everything is fine. No leak, no black IPL.

Comparing the two codes to convert BitmapToIpl I can notice:

• yours : based on GetDIBits, in one call
• The other code : based on a Scanline of the Bitmap that copies the data to the rows of IPL, row by row.

Windows documentation states that GetDiBits data is aligned to DWORD. By the way, GetDiBits returns an error level message that is not processed in the call. Opencv Documentation states that data in IPL is not necessarilly contiguous from one line to next one, some data alignment being as well applied. The widthstep must be used to switch from one line to next one.

So, I think the current routine of your wrapper is wrong in handling "special cases" where data alignment is not fitting in the two cases, or when IPL data is not contiguous from one row to the next.

I would suggest to use the row by row thechnique, based on this code. Pay attention, it does not allocate the Ipl and runs as a procedure. It has to be adapted not to break compatibility. Carrefull as well, the routine "plays" with origine of data IPL_ORIGIN_BL, and this may impact the vertical flip of the image.

It must be "tested" so the two routines returns identical IPL header and "look" the same on display.

Drop as well the lazarus portion, I think it is not working as well, same reason : block copy. {----------------------------------------------------------------------------- Procedure: Bitmap2IplImage Author: G. De Sanctis Date: 15-may-2007 Arguments: iplImg: PIplImage; bitmap: TBitmap Description: create a new IplImage and convert a Windows bitmap (24 bit) to it -----------------------------------------------------------------------------} procedure Bitmap2IplImage(iplImg: PIplImage; bitmap: TBitmap); VAR dataByte : PByteArray; RowIn : pByteArray; {$ifdef MSEGUI} offset : longint; size : sizety; sh : integer; po2 : prgbtripleaty; j,k : integer; {$endif} {$ifdef LAZARUS} lazImg: TLazIntfImage; {$endif} BEGIN TRY assert((iplImg.Depth = 8) and (iplImg.NChannels = 3), 'Not a 24 bit color iplImage!'); // MSEGUI NOT defined {$ifndef MSEGUI} assert((bitmap.pixelFormat = pf24bit) , 'Not a 24 bit color BMP bitmap!'); {$endif}

iplimg.Origin := IPL_ORIGIN_BL;
iplImg.ChannelSeq := 'BGR';

{$ifdef MSEGUI}
po2:= bitmap.scanline[0]; offset := longint(iplimg.ImageData); // scan every line from IPL image because could have filler // bytes in the end FOR j := 0 TO bitmap.size.cy - 1 DO BEGIN offset := longint(iplimg.ImageData) + iplImg.WidthStep j; FOR k := 0 TO bitmap.size.cx - 1 DO BEGIN dataByte := pbytearray( offset); with po2^[(bitmap.size.cy - 1 - j)bitmap.size.cx + k] do begin dataByte[0] := blue; dataByte[1] := green; dataByte[2] := red; end; offset := offset + 3 ; END; END; {$else} {$ifdef LAZARUS} lazImg := TLazIntfImage.Create(bitmap.Width, bitmap.Height); lazImg.LoadFromBitmap(bitmap.Handle, bitmap.MaskHandle); RowIn := lazImg.GetDataLineStart(bitmap.height -1 ); {$else} RowIn := Bitmap.Scanline[bitmap.height -1 ]; {$endif} dataByte := pbytearray(iplimg.ImageData); {direct copy of the bitmap row bytes to iplImage rows} CopyMemory(dataByte, rowin, iplimg.ImageSize); {$endif} {$ifdef LAZARUS} lazImg.Free; {$endif} Except on E: Exception do begin raise Exception.Create('Bitmap2IplImage- error - ' + e.Message); end; END END; {Bitmap2IplImage}

[blaisexen]

Hi,

I have also experienced this kind of issue, and found out that you have to create first the TBitmap, Assign a Bitmap image to the TBitmap, Assign an Image Size for the IplImage then you call BitmapToIplImage.

like this:

bmpFace := TBitmap.Create; //Creat the TBitmap
bmpFace.Assign(theimage); //Assign image to TBitmap
dbFaces := cvCreateImage(cvSize(iwidth,iheight), 
IPL_DEPTH_8U, 1{for grayscale, 3 for RGB}); //Assign size to IplImage
dbFaces := BitmapToIplImage(bmpFace);//You got now the IplImage

I'm sorry if I got it wrong, Good Luck

[HuguesDug]

Hello @blaisexen

Nice to read your comment. But I think you got it wrong :-)

First, your code will leak. You create a PCvIplImage and allocate memory to it via CvCreateImage. Then you assign the result of the call to conversion routine to the same pointer. But there is an additionnal CvCreateImage inside the routine. Allocated data in the first call is lost and you will leak.

By the way, I am discribing a random problem (1% of the cases out of a run on thousands of images).

So far, I stick to my point : data alignment is not necessarilly the same in DataSource of Ipl and in GetDiBits. OpenCv may fill end of line with unknown quantity of bytes, that makes impossible to transfert the whole memory block in one go.

This is why they give the "widthstep" to skip from one row to the next.

The only reliable solution is to copy lines of the bitmap to lines of the data.

Hugues

[blaisexen]

Ok, But if you copy lines of bitmap to lines of data it will be so slow I think! thanks for the reply

[Laex]

The fastest operation is nop

[blaisexen]

@Laex Ok, so do you think @HuguesDug was right?

@HuguesDug can you realign my code to make it no leak?

bmpFace := TBitmap.Create; //Creat the TBitmap
bmpFace.Assign(theimage); //Assign image to TBitmap
dbFaces := cvCreateImage(cvSize(iwidth,iheight), 
IPL_DEPTH_8U, 1{for grayscale, 3 for RGB}); //Assign size to IplImage
dbFaces := BitmapToIplImage(bmpFace);//You got now the IplImage

I hope to get the result, thank you

[HuguesDug]

@Laex : I like the comment :-)

@blaisexen : it will not be slower than what does getDiBits. You can do math pointer on operation, and probably do a parallel for to increase speed. You can as well do a memcopy per line I guess.

Your code can be changed in this way: Var bmpface:TBitmap; dbfaces:PiplImage; Begin bmpFace := TBitmap.Create; //Create the TBitmap bmpFace.Assign(theimage); //Assign image to TBitmap dbfaces:=BitmapToIplImage(bmpFace);//You got now the IplImage

{Here do what you need to do}

CvReleaseImage(dbFaces); bmpFace.Free;

It will work... with limitations I indicated above in the comments. In some case, with unaligned/not contiguous data inside the IplImage, you will have a black image or a leak or whatever....

[HuguesDug]

Hello,

Continuing to work on the topic, I may have a different explaination for the failure that I am going to investigate...

When I end up with "black" IPL, the GetDIBits returns 0, meaning that the function did not work. The explaination given on MSDN states that it could be caused by the use of a Bitmap Handle not being DIB but Memory DC.

Reading the code, although the error checking is not done in value returned by GetDiBits, I do not see any line that ensure that the Bitmap.handle is actually a DIB... I think this problem is not existing in the other routine I provided, as we copy memory.

Probably, it is a better explaination for the random failure of the routine.

[HuguesDug]

Hello,

To close the point, here is the fix to apply. I tested it under stress, with data from a "bitmap from file" as well as with data from a "DirectX grabber". With proposed fixed, no more issues.

Var BMI: BITMAPINFO; MemDC:Integer; begin Assert(bitmap.PixelFormat = pf24bit); // Ïîêà òîëüêî òàêîé ôîðìàò ZeroMemory(@BMI, sizeof(BMI)); BMI.bmiHeader.biSize := sizeof(BITMAPINFOHEADER); BMI.bmiHeader.biWidth := bitmap.Width; BMI.bmiHeader.biHeight := -bitmap.Height; BMI.bmiHeader.biPlanes := 1; BMI.bmiHeader.biBitCount := 24; // only 24-bit BMI.bmiHeader.biCompression := BI_RGB; Result := cvCreateImage(cvSize(bitmap.Width, bitmap.Height), IPL_DEPTH_8U, 3); // only 24-bit memDC:=CreateCompatibleDC(0); SelectObject(memDC,bitmap.Handle); GetDIBits(memDC, bitmap.Handle, 0, bitmap.Height, Result^.ImageData, BMI, DIB_RGB_COLORS); DeleteDC(memDC); end;

[blaisexen]

@HuguesDug nice one!,

can you share your Ocv.Utils.pas?

So I may update what I have in here?

OPS, sorry I see it in the update now, thank you

[HuguesDug]

This one was really tricky.

I saw another one for you (in components)... but it will be another day. Some important tasks to conduct on the short term.

[blaisexen]

Wow, that's so good! thanks in advance! :dancer: