Different stereo cameras installed in camera box 2018-01-25

[NewcombMaria]

After yesterday's hardware test for the stereo-vis y-axis velocity problem that has been ongoing during this current season 5, we determined that the problem has been caused by a fault with one or both of the stereo cameras. Today Stuart, John, and I will replace both of the stereo cameras with two that had formerly been on the gantry side-arm, keeping the original lenses but replacing the cameras.

We will update the metadata, adjust the camera settings, and go through all procedures necessary to ensure data quality. @dlebauer (and/or anyone: @JeffWhiteAZ) please provide advice/input to ensure that we complete all steps necessary for replacing instruments. Please add/comment on the bullets below:

• metadata changes: date of installment, serial numbers, we'll check if the camera models are the same or different

• they will be mounted in the same position as the previous cameras. Are there alignment tests that need to be completed?

• camera settings related to images: gain, gamma, white balance, exposure, image size and format

• camera settings related to network connection

[max-zilla]

@craig-willis FYI we should have this on radar.

[max-zilla]

This is the stereo RGB current fixed metadata: https://github.com/terraref/sensor-metadata/blob/master/sensors/stereo/sensor_fixed_metadata.json

@craig-willis @dlebauer and I should discuss. we need to adjust this fixed JSON object to support by-date references so we can route the dataset to correct sensor metadata based on when the dataset was captured

[max-zilla]

@smarshall-bmr should make a pull request with edits to this XML file: https://github.com/terraref/lemnatec-fixed-metadata/blob/master/stereoCameraBox.xml

we will then update the JSON file and code to reflect these changes (supporting multiple date ranges/multiple calibrations)

[NewcombMaria]

Latest update: it turns out that the installation of the 2 alternative cameras did not result in a sustainable fix. After time on the afternoon of 25 January the scan even with the new cameras faulted by server problems. For a while today Stuart tried one old camera and one new camera and updated camera settings and it was working well for a while but just recently the server drops returned. For now the camera re-installation is on hold and it's uncertain which cameras will be selected for the long run.

[NewcombMaria]

Data from the afternoon of 2018-01-24 through mid-day 2018-01-29 will need careful attention to edit metadata for data that should be kept, especially 2018-01-27, 01-28, and morning of 01-29 which are days/times with good data but with the serial number of one of the stereo cameras incorrect which will need to be edited in metadata. Some times/days the data will need to be noted as ERROR when we were running tests for diagnostic testing only and the camera lenses didn't match up.

Starting today at 11:45 AZ time, (2018-01-29 11:45), the original cameras in original positions were connected and are currently running to match with the metadata serial number settings. It appears likely that cameras were not the problem. Markus Radermacher (LT) provided a software revision that initiates image fetching in the software and disabled all wago IO device interaction and so far today Stuart is running the scan at optimal speed (0.33 m/s) without server drops and without losing images.

[dlebauer]

@NewcombMaria so ... in the end the cameras were not switched? can we close this, or is there anything else we need to do?

[NewcombMaria]

@dlebauer the only remaining item that needs follow-up is that the data and metadata from the afternoon of 2018-01-24 through mid-day 2018-01-29 needs either metadata correction for some data-sets or for other data-sets the data itself needs 'ERROR' notation.

Some of the scans that were run in that time-frame generated data that should either be deleted or marked clearly by "ERROR" (for example when one camera in the stereo pair had one type of lens and the other stereo-paired camera had a different lens). However the data collected on 2018-01-27 and 2018-01-28 and the morning of 2018-01-29 are reasonable data but need notation in the metadata that one of the two stereo cameras had a different serial number. I'll need to find out the correct serial number and if it was the south or north camera.

Will update with more details soon.

[dlebauer]

@NewcombMaria thanks! if you could just provide a bulleted list of what needs to happen for what date ranges, we can implement.

[NewcombMaria]

@dlebauer here's a list by dates for StereoTop data files from StereoVis scans. Let me know if you have follow-up questions:

• 2018-01-27 and 2018-01-28 and only the file directories before 11:45 on 2018-01-29. The data files are usable and are good data but metadata needs to be revised to change the serial number of the North camera. It would be best if the file directories could be flagged by 'Warning' to inform that one of the two stereo cameras was changed and there could be minor issues with alignment or other detectable differences. The North camera during these 2.5 days was SN: 02-2623A-07137 (NOT 02-2623A-07141)

• Starting after 11:45 on 2018-01-29 (including the afternoon scan on the 29-Jan) and all dates forward. The North camera was switched back to the original camera with the original serial number (SN: 02-2623A-07141)

• 2018-01-24 from 09:07 to 12:54. During this time on the 24-Jan the file directories need to be flagged by 'Warning', since there were some stops and starts to change the y-axis velocity and also at some time point there was a switch to different server software that Markus R. (LT) provided that resulted in gaps in the field with no images (10 to 15 meter sections along the y axis).

• Starting 2018-01-24 at 14:22 through the end of the day on 2018-01-26. All data files should be marked with "ERROR" and should not be processed. These days/times we were running tests with different cameras in positions off-set from normal and with different lenses, and it's best to not process these data from the afternoon of 24-Jan through the end of the day 26-Jan.

• Markus occasionally ran his test scan on a few dates in December and January but is test script is clearly named something like markus_test.