search

ev3dev / ev3dev

ev3dev meta - bug tracking, wiki and releases
http://www.ev3dev.org
GNU General Public License v2.0
634 stars 85 forks source link

Meaning of raw values from the EV3 color sensor #1386

Closed JakubVanek closed 4 years ago

JakubVanek commented 4 years ago

Is your feature request related to a problem? Please describe. No, it just stems from my curiosity.

Describe the solution you'd like Adding documentation about the raw modes of the EV3 color sensor (in https://github.com/ev3dev/lego-linux-drivers/blob/4cddeee8d48f3fc01dff3be59906c9a5a2c1c635/sensors/ev3_uart_sensor_defs.c#L87)

Describe alternatives you've considered N/A

Additional context I have been playing with c4ev3/EV3-API and there is a currently a new userspace sensor API in development. I wanted to know what the RGB-RAW and REF-RAW values mean and I think I found the answer. REF-RAW: https://github.com/simonedegiacomi/EV3-API/commit/20a57f7b944a1e53477ebdfe61f96ebf2d6c66b7#r37442226 RGB-RAW: https://github.com/simonedegiacomi/EV3-API/commit/8562bd29da2ab430807741f7374ac60434eb793e#r37442413 (followup to REF-RAW)

However perhaps the documentation of the raw values should be put on its own page (if included with all the details), as it may be too much information for one sensor/mode.

I haven't proven if the values comes directly from the ADC by disassembling the sensor firmware, but I don't think there's lot the sensor could do with the values.

I also have an idea about what the COL-CAL mode does and why it behaves like it does. However, in this case, I have no proof of whether this is true. I think the mode is used in factory in order to provide calibration values for the COL-REFLECT, COL-AMBIENT and COL-COLOR modes to work correctly. Therefore, the sensor waits for extra data to come after the mode switch occurs and this makes the keepalive time out. I am also afraid a little to do experiments with sending random data to the sensor, as that could erase the existing data.

Full proof would be the full disassembly of the color sensor firmware. Some years ago, I have downloaded the flash/eeprom contents from the sensor. I currently do not have time to disassemble it, but I can send it via email if anyone wants to play with it.

Best regards,

Jakub Vanek

JakubVanek commented 4 years ago

Oh, I have now discovered that there exists a direct attribute in ev3dev and a INPUT_WRITE opcode in the stock firmware and that the stock firmware has some examples with strings like LEGO-FAC-CAL-1. Perhaps the COL-CAL mode is an obsolete variant of that or the calibration is done differently there.

JakubVanek commented 4 years ago

Hmmm, switching to COL-CAL mode seems to enable the calibration command...

https://github.com/mindboards/ev3sources/blob/78ebaf5b6f8fe31cc17aa5dce0f8e4916a4fc072/lms2012/lmssrc/Test/Test.lms#L1904

It also appears that calibration parameters can be somehow read from the sensor.

JakubVanek commented 4 years ago

Suprisingly, searching through the firmware binary for LEGO-FAC-CAL-1 does not find anything. Perhaps the firmware detects it with some hash or it is removed in final sensor revisions.

dlech commented 4 years ago

Nice find. I'm surprised I never spotted these tests in the source code before.

I'm pretty sure the RAW modes are just the raw ADC value since 0-1023 sounds suspiciously like a 10-bit value.

It looks like we can update the docs to say that the CAL modes on the gyro sensor and color sensor return float values and only the first 3 out of the 4 values are interesting since those are the ones printed on the screen in the test.

Based on the way I read the program, I'm guessing that the color sensor is held up to white, then the calibration is started. It prints the before and after calibration values. I'm not really sure what "backward" and "forward" would be for the gyro calibration though.

JakubVanek commented 4 years ago

If the calibration command returns three 32bit entries, then the contents of the color sensor's EEPROM make more sense. It is all zeros, except for first 12 bytes:

$ od --width=4 -Ad -t x1 color-eeprom.bin 
0000000 00 00 05 12
0000004 00 00 04 4b
0000008 00 00 06 71
0000012 00 00 00 00
*
0000640
laurensvalk commented 4 years ago

Nice finds @JakubVanek. I've been wondering about this mode for a while, but all I could read were zeros.

Hmmm, switching to COL-CAL mode seems to enable the calibration command... It also appears that calibration parameters can be somehow read from the sensor.

It looks like we can update the docs to say that the CAL modes on the gyro sensor and color sensor return float values and only the first 3 out of the 4 values are interesting since those are the ones printed on the screen in the test.

I don't seem to be able to read any values other than 0, even when writing LEGO-FAC-CAL-1 to direct and reading bin_data.

As I'm reading this issue --- I'm starting to wonder, did anyone actually get any nonzero values from the sensor though?

laurensvalk commented 4 years ago

@JakubVanek - if you break your sensor in the process of trying, I'll send you a replacement :)

JakubVanek commented 4 years ago

Thanks!

Luckily, I will not need a replacement sensor - I found an old disassembled one that I had. I had success with recalibrating this sensor (changing the value scale) with the Test.rbf application on the stock lms2012 firmware. However, it needs a "trick" with starting the calibration quickly.

The calibration values are not in the range 0~1023 - for example, when calibrating the sensor when it was pointing into the air, I got 4595 3787 4755. When I point it on a white paper, I get 660 659 654. However these values are skewed by the fact that without the sensor casing, the color LEDs illuminate the photodiode directly.

I don't know yet what these values mean, I will try to investigate it. It seems that they could be fixed-point coefficients for multiplying the raw measured values.

The zeros seem to get shown when the sensor times out (as discovered by @dlech), restarts and switches to a different mode. I wonder if this could be a safety measure by LEGO to prevent accidental recalibrations.

On ev3dev, this should work as well. Writing COL-CAL to mode and then quickly writing LEGO-FAC-CAL-1 (without newline) to direct should do the same thing, if the low-level differences between ev3dev and lms2012 do not interfere with this.

Best regards,

Jakub

laurensvalk commented 4 years ago

On ev3dev, this should work as well. Writing COL-CAL to mode and then quickly writing LEGO-FAC-CAL-1 (without newline) to direct should do the same thing, if the low-level differences between ev3dev and lms2012 do not interfere with this.

I tried this, but maybe I wasn't fast enough.

I had success with recalibrating this sensor (changing the value scale) with the Test.rbf application on the stock lms2012 firmware. However, it needs a "trick" with starting the calibration quickly.

Nice! Is it also possible to read the current values without overwriting them with the calibration routine?

JakubVanek commented 4 years ago

On ev3dev, this should work as well. Writing COL-CAL to mode and then quickly writing LEGO-FAC-CAL-1 (without newline) to direct should do the same thing, if the low-level differences between ev3dev and lms2012 do not interfere with this.

I tried this, but maybe I wasn't fast enough.

I'm trying to reimplement this on C4EV3 and I too cannot get it to work, so perhaps there is more to triggering the calibration.

I had success with recalibrating this sensor (changing the value scale) with the Test.rbf application on the stock lms2012 firmware. However, it needs a "trick" with starting the calibration quickly.

Nice! Is it also possible to read the current values without overwriting them with the calibration routine?

I don't think it is possible to do this with LEGO-FAC-CAL-1, but it may be possible with a different command (but I don't know if such a command exists).

JakubVanek commented 4 years ago

The following (pseudo-)code seems to work reliably on hacked C4EV3:

printf("switching to COL-CAL...\n");
ReadEV3ColorSensorCAL(IN_2, &rgba);

printf("sending calibration challenge...\n");
SendEV3ColorSensorCAL(IN_2);

Wait(500);

printf("reading COL-CAL data...\n");
ReadEV3ColorSensorCAL(IN_2, &rgba);
printf("R %6d G %6d B %6d A %6d\n", rgba.red, rgba.green, rgba.blue, rgba.ambient);

printf("switching to COL-REFLECT...\n");
ReadEV3ColorSensorLight(IN_2, ReflectedLight);

Apparently the sensor needs some extra time to return the calibration data - without the 500ms (or 1000ms in the Test.lms program) delay the calibration data don't print properly.

Example output:

initializing port...
switching to COL-CAL...
sending calibration challenge...
reading COL-CAL data...
R    747 G    710 B    702 A      0
switching to COL-REFLECT...
done!

However I suspect there is a bug in ev3dev's sensor definitions. The COL-CAL mode has data_type set to LEGO_SENSOR_DATA_S32; however the values above were read with the following code:

rgb->red        = ((uint8_t)data[0]) + (((uint8_t) data[1]) << 8u);
rgb->green      = ((uint8_t)data[2]) + (((uint8_t) data[3]) << 8u);
rgb->blue       = ((uint8_t)data[4]) + (((uint8_t) data[5]) << 8u);
rgb->ambient    = ((uint8_t)data[6]) + (((uint8_t) data[7]) << 8u);

I think LEGO_SENSOR_DATA_U16 suits this more. I have also received 65535 max, which agrees with the LEGO's typedata.rcf.

JakubVanek commented 4 years ago

Hi all,

I have finally gotten to reverse engineering the color sensor firmware. This is what I've found with regards to this issue:

Full results can be found at https://github.com/JakubVanek/lms2012-stuff/tree/master/ev3color . It is not yet fully complete, I need to to refine the pseudocode so it looks more like plain C (rename offsets to variables, reconstruct switches from jumps, etc.).

Best regards, Jakub

JakubVanek commented 4 years ago

I have realized that it is possible to get an approximation of the red calibration coefficient indirectly. This is because COL-REFLECT sends calibrated data and REF-RAW sends raw data. I think that the ratio of the COL-REFLECT reflectivity and the difference between REF-RAW subvalues should remain constant.

It might be possible to roughly approximate coefficients for other colors as well. The only option I see is to go through the COL-COLOR mode. It would be necessary to use a surface and sensor position combination such that the color detected by COL-COLOR is unstable. By finding the threshold that causes the instability, one could guess the calibrated non-red color reflectivity. When combined with a RGB-RAW measurement, the resulting ratio should again get near the calibration coefficient.