LEGO EV3 Pixy2 Camera block returns wrong data for x or y >= 127?

[IAssemble]

Hi,

When using the LEGO "Pixy2 Camera" block from the EV3 software to read RGB colors from the PIXY2, I suspect that any x or y co-ordinate >= 128 is probably truncated to 127 before the pixel data is read since scanning the entire region from 0..255 in both directions results in sensible data for only the top left quadrant (I have reported this to support@pixycam.com with an example program and image). The rest of the data looks like it is probably obtained from the pixel with corresponding x or y co-ordinate being saturated to 127 because there are horizontal and vertical lines extending from the top left quadrant.

One guess is that somewhere the co-ordinates are being treated as signed 8 bit numbers rather than unsigned when converting from the input to the LEGO block.

I have looked at the code and cannot see where this saturation (or other error) may be occurring but it is difficult to read the code in the VIx files in the LEGO block itself so perhaps I have missed something there?

Also, I think even when this is resolved there may be a minor issue that the co-ordinates 0..255 are being scaled slightly incorrectly by the following lines in lego_getData():

        x = buf[0]*CAM_RES2_WIDTH/255;
        y = buf[1]*CAM_RES2_HEIGHT/255;

I suspect these should be:

        x = buf[0]*(CAM_RES2_WIDTH-1)/255;
        y = buf[1]*(CAM_RES2_HEIGHT-1)/255;

although this does not explain the main issue about co-ordinates >- 128 being used incorrectly.

Also, I am a little curious why the co-ordinates are scaled from ranges 0..255 in both x and y to access the image data which is rectangular rather than square. This changes the aspect ration of the sampled image and is less intuitive to appreciate how to determine which co-ordinates correspond to which prtions of the image (for example if using the co-ordinates displayed when hovering the mouse over part of an image in the PixyMon application.

Thanks IAssemble

[richlegrand]

Hi David, This is indeed a bug on our end. We're working to resolve asap.

thanks, --rich

[IAssemble]

Hi Rich,

Many thanks for the update. Please let me know if you'd like any help investigating or testing.

Thanks David

[IAssemble]

HI Rich,

If it helps, I did a little more investigation (including disassembling a .rbf file containing a call to "Pixy2 Camera" in RGB measure mode). It appears that the code generated for the conversion of X and Y co-ordinates uses the MOVEF_8 function to reduce the co-ordinates to bytes. The implementation of this is for signed 8 bit values so the value is truncated to the range -128 thru 127. The whole byte is then passed right through to the PIXY2 camera firmware. So I can work-around the issue with the current code by transforming the X and Y co-ordinates like this before passing them to the"Pixy2 Camera" block: ((X+128) % 256 - 128) This transforms the co-ordinate to a value in the range -128 thru 127 where an input of 128 thru 255 is mapped to -128 thru -1 so gets passed as the same bit pattern as it would if it were an unsigned byte.

I have now managed to capture this image using this workaround in my EV3 program (PixyMon image on left and photo of EV3 screen on right):

I look forward to an updated "Pixy2 Camera" block so I can capture images like this without the workaround :)

Many thanks David

[IAssemble]

Hi Rich,

And of course the conversion from RGB to monochrome will be much better when issue #16 is addressed too ;-)

Thanks David

[richlegrand]

Nice! That's an impressive amount to skill to disassemble and fix like that. :) :)

I'm waiting for my LEGO developer to get back from vacation. I assume it should be a quick/easy fix. I'll keep you posted.

thanks, --rich

On Wed, Aug 14, 2019 at 5:17 PM David Gilday notifications@github.com wrote:

HI Rich,

If it helps, I did a little more investigation (including disassembling a .rbf file containing a call to "Pixy2 Camera" in RGB measure mode). It appears that the code generated for the conversion of X and Y co-ordinates uses the MOVEF_8 function to reduce the co-ordinates to bytes. The implementation of this is for signed 8 bit values so the value is truncated to the range -128 thru 127. The whole byte is then passed right through to the PIXY2 camera firmware. So I can work-around the issue with the current code by transforming the X and Y co-ordinates like this before passing them to eht "Pixy2 Camera" block: ((X+128) % 256 - 128) This transforms the co-ordinate to a value in the range -128 thru 127 where an input of 128 thru 255 is mapped to -128 thru -1 so gets passed as the same bit pattern as it would if it were an unsigned byte.

I have now managed to capture this image using this workaround in my EV3 program (PixyMon image on left and photo of EV3 screen on right): [image: pixy2-EV3-3] https://user-images.githubusercontent.com/6344351/63060148-538e4180-bee9-11e9-864e-b095081575cc.png

I look forward to an updated "Pixy2 Camera" block so I can capture images like this without the workaround :)

Many thanks David

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/charmedlabs/pixy2/issues/17?email_source=notifications&email_token=AAG66DOJX4SCWIDYPHNGRXDQESAAVA5CNFSM4IJE7QMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4KI4SA#issuecomment-521440840, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG66DN7DUDZN7GHPBLY5ETQESAAVANCNFSM4IJE7QMA .

-- Charmed Labs www.charmedlabs.com

[IAssemble]

Thanks :) David

[richlegrand]

Hi David, Please find the attached files. It attempts to fix the issues you raised:

1) The x and y inputs to RGB mode range between 0-315 for x and 0-207 for y 2) A saturation boolean arg is added to RGB mode

You will need to update your Pixy's firmware to support the saturation flag.

Let me know if you have any questions.

thanks, --rich

On Thu, Aug 15, 2019 at 2:45 AM David Gilday notifications@github.com wrote:

Thanks :) David

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

-- Charmed Labs www.charmedlabs.com

[richlegrand]

Here's the link: https://drive.google.com/file/d/18Z0cYl0-3j0k_37cHDIyPsZ9JPkFp6b1/view?usp=sharing

[IAssemble]

Hi Rich,

Brilliant! :-)

This version appears to resolve the x/y co-ordinate truncation issue (and preserves the camera aspect ratio) and adds support for unsaturated RGB as discussed.

Here are example photos of the EV3 screen of images captured using my test program to demonstrate it working. As you can see, there is a marked difference between using saturated and unsaturated RGB when converting to a monochrome image in this way.

Many thanks! David