NHMDenmark / Herbarium-Sheets-workstation

Workstation and workflows for herbarium sheets for mass digitisation (DaSSCo)
0 stars 0 forks source link

Decide what extra we need to calibrate at beginning of session #19

Closed PipBrewer closed 5 months ago

chelseagraham commented 7 months ago

An analogous program responded with the following when asked how exactly they calibrate at the beginning of each session what post-processing they carry out


Our colour management is quite basic, in part because with preserved plants the original colour isn't maintained. Every specimen has a golden thread colour chart included so that if we wanted to we could do more complex colour correction at a point in the future.

At the start of the session we take a test shot and check the white balance by hovering over patch 13 (3rd darkest grey patch)to check the RGB values don't deviate by more than 1 or 2 values. I created a custom white balance using patch 13 and embedded it in a custom-made style (collection of settings applied to each image) used in Capture One Pro, which we use for tethered imaging.

To check image exposure we hover over the white value in the test shot to make sure it's ~ 230 (i.e. not over exposed). We used to also check the black patch too (when using an x-rite chart with our older set-up which used Bowens fluorescent tube continuous lighting) ensuring it was around 30. When we switched to LED lightboxes the LEDs didn't render the black patch correctly (they weren't photography LEDS, we now have setups with LEDS appropriate for photography) so we couldn't use it as a reference.

We also correct for light fall off as our lightbox is open at one side by producing a Lens Cast Calibration setting using a LCC plate (The DTCH Digitization Workflows Guide has a good section on setting this up and creating bespoke ICC Profiles).

We use Adobe RBG colour space for both RAWS and TIFFS and Capture One's default ICC profile and film curve for the camera model we have. We do not own any colour management software either for bespoke ICC profile creation or for colour QA.


chelseagraham commented 7 months ago

Another program has said


For herbarium imaging you will probably find that its unnecessary to colour manage everyday in order to get good results (its very time consuming so there is a trade-off to be had), but that there are a few key things within the capture software settings that will probably greatly improve your golden thread results with a bit of experimentation with settings.

you want to try to get the colour as good as possible as you only really want to be doing this once! Depending on the number of specimens to be imaged, there is a balance to be had though, as colour calibration can be time consuming so we tend to use the Golde thread to analyse the stations on setup and periodically to monitor the stations, but it isn’t feasible (and probably unnecessary) to calibrate every session/daily, as we find that we can get fairly good colour fidelity without this. Thanks for forwarding your manual, I will read this properly next week but I can see from a quick scan that you use DeltaE for analysis of the targets, we use the GoldenThread software that comes with the GoldenThread targets but they are essentially measuring the same parameters and the targets are also very similar.

The lighting will definitely have an impact on the colour too, but something you could try is an LCC profile (this may be covered somewhere else in your manual so apologies if so!) you may also find that the curve being used may need adjusting? I am not familiar with Phocus software but I will take a look! Another thing to keep in mind is the 3D nature of the herbarium material and that it will behave slightly differently under the lighting than paintings/books (getting double shadows etc). We also check for light uniformity using a white board as you do, and also use this for creating LCC profiles.

As you are using CaptureOne I would strongly recommend using the Linear Response curve under the Base Characteristics if you aren’t already, as this will should improve the colour results (you may need to up the lighting a little). If you’re not already using this, I would try it before going down the LCC route.

LCC is fairly easy to set up (just need a clean white board) and can be useful if your lighting and camera setup are strictly maintained in the same location.


chelseagraham commented 6 months ago

Held a Teams meeting with a colleague from a herbarium mass digitization project on 15 february. They suggest

chelseagraham commented 6 months ago

@Gomismis and/or @JesperMJensen tomorrow, 28 February, please take a series of pictures of the large Device Level target (two each for the following scenarios) at various exposures and angles, as follows

1/15th of a second shutter speed - flat(2) 1/20th of a second shutter speed - flat (2) 1/30th of a second shutter speed - flat (2)

1/15th of a second shutter speed - at a 10 degree angle (2) 1/20th of a second shutter speed - at a 10 degree angle (2) 1/30th of a second shutter speed - at a 10 degree angle (2)

1/15th of a second shutter speed - at a 20 degree angle (2) 1/20th of a second shutter speed - at a 20 degree angle (2) 1/30th of a second shutter speed - at a 20 degree angle (2)

1/15th of a second shutter speed - at a 30 degree angle (2) 1/20th of a second shutter speed - at a 30 degree angle (2) 1/30th of a second shutter speed - at a 30 degree angle (2)

Please look at the guide here: SCI-SNM-DigitalCollections/DaSSCo/Workflows and workstations/GUIDES/Herbarium/HerbariumImaging/ for updated settings to the camera.

Please apply edits from the guide ( 56, 57, and 58 ) to the second image of each pair and keep the first totally raw before exporting Please rename the files with _flat115, _flat120, _flat130 _10degree115, _10degree120, _10degree130 _20degree115, _20degree120, _20degree130 _30degree115, _30degree120, _30degree130

and

_flat115,edit _flat120edit, _flat130edit _10degree115edit, _10degree120edit, _10degree130edit _20degree115edit, _20degree120edit, _20degree130edit _30degree115edit, _30degree120edit, _30degree130edit

Please export to TIFF

Please save the files here: N:/SCI-SNM-DigitalCollections/DaSSCo/Pilot Data/Herbarium Working Data/20240228/DeviceLevelTarget

chelseagraham commented 6 months ago

@Gomismis and/or @JesperMJensen tomorrow, 28 February, please take a series of pictures of 10 herbarium sheets (two each for the following scenarios)

1/15th of a second shutter speed 1/20th of a second shutter speed 1/30th of a second shutter speed

Please look at the guide here: SCI-SNM-DigitalCollections/DaSSCo/Workflows and workstations/GUIDES/Herbarium/HerbariumImaging/ for updated settings to the camera.

Please apply edits from the guide ( 56, 57, and 58 ) to the second image of each pair and keep the first totally raw before exporting Please rename the files with the following ending appended for each _115, _120, and _130

and

_115edit, _120edit, and _130edit

Please export to TIFF

Please save the files here: N:/SCI-SNM-DigitalCollections/DaSSCo/Pilot Data/Herbarium Working Data/20240228/LinearCurveComparison

Gomismis commented 6 months ago

We managed to take all the required pictures and I have transferred them to the N-drive. When i have renamed them I will share the path in this thread.

Gomismis commented 6 months ago

all pictures are renamed and saved here:

N:\SCI-SNM-DigitalCollections\DaSSCo\Pilot Data\Herbarium Working Data\20240228\LinearCurveComparison

chelseagraham commented 6 months ago

Created the Device Level Target folder and moved TIFF exports there

chelseagraham commented 5 months ago

Kim does not suggest we apply the Linear Response Curve in Capture One. From email

Re: Test data from Herbarium C.pdf

CG: LINEAR RESPONSE CURVE Summary: In our meeting, we discussed how Laura Green of Kew suggested that we employ the Linear Response Curve (https://support.captureone.com/hc/en-us/articles/360002424317-What-is-the-Curve-section-of-the-Base-Characteristics-tool) to our images before exporting from Capture One. She said that Kew uses it and it is a cultural heritage standard. I see this curve is suggested in the Digital Transitions Color Reproduction Guide (https://heritage-digitaltransitions.com/wp-content/uploads/woocommerce_uploads/2018/12/DTCH-ColorReproductionGuide_2018.pdf) for most flat art reproduction photography. They say, “The linear response curve is linear throughout most of the tonal range, but provides slight compression f bright highlights that preserves tone and color in highlights.” it provides a slight compression of the bright highlights Task: https://github.com/NHMDenmark/Herbarium-Sheets-workstation/issues/19#issuecomment-1966659985 Matilde and Jesper imaged 10 herbarium sheets at different shutter speeds 1/15 second (over-exposed), 1/20 second and 1/30 second. They took two images for each – one that they edited using this curve and one that they did not. This way we have examples of before and after and also to ability to see how the curve reacts to an image with exposure warnings. Their data is located here: N:/SCI-SNM-DigitalCollections/DaSSCo/Pilot Data/Herbarium Working Data/20240228/LinearCurveComparison Feedback Requested: Do you agree that digitizers should apply this curve in Capture One before exporting?

KSP: No I don’t think we should. I am very suspicious of this linearization done by Capture One. I suspect this is either histogram equilization or gamma correction and both actually alters the pixel values in a non-linear way that may cause loss of information (the process cannot be inverted). If you look at the histograms of each color channel it does a different transformation on each channel which in effect changes the colors in each pixel. So since we do not fully understand how this is computed I would say no to this. We want the raw originals to be changed as little as possible.

The linear images looks nice, but we can do this afterwards in processing on the servers for the web friendly nice version of the original image, if we want to.

The 1/15 images below are overexposed and this cannot be fixed by the linear transform. I checked the histograms and did some statistics. You find an excel sheet and pdfs of color histograms in N:/SCI-SNM-DigitalCollections/DaSSCo/Pilot Data/Herbarium Working Data/20240228/LinearCurveComparison/Kims_Stats

An image is overexposed if there are many pixels that are saturated at the maximum value that can be represented in each channel at a pixel. In our case the camera uses 16 bit unsigned integers which means the maximum value in each channel is 65535 (=2**16-1). If RGB all have maximum value this means the pixel is white. Similar an image is underexposed if there are many black pixels (that is RGB all have value 0) and the color histograms are shifted towards small values. In our case we have trouble with some of the shutter speed settings leading to overexposure.

Check out the columns in the spreadsheet called “#Saturated pixels in …” and especially the one called max (org and lin refers to the original and the linear transformed version called edit in file names). For the 1/15 images you can see a high max count and you can even see it in the histograms as peaks at the max value of 65535

The 1/20 and 1/30 images are also overexposed but slightly less with in the order of 10 pixels in the 1/30 images hitting the maximum value. Looking at the histograms the total range of pixel values are better used (spread out through the complete range) when using 1/30 speed. Ok, there are a few cases where it looks like 1/20 is slightly better such at image NHMD_Herb01_10.

We should at least use the 1/30 shutter speed setting. Can the camera do a faster shutter speed? E.g 1/40?

chelseagraham commented 5 months ago

After discussion with Kim, we will image a white sheet before each session to assess if anything is shifting with illumination. Please see https://github.com/NHMDenmark/Herbarium-Sheets-workstation/issues/86

We will continue to image the Device Level Target flat