Ultimaker / Cura

3D printer / slicing GUI built on top of the Uranium framework
GNU Lesser General Public License v3.0
6.18k stars 2.08k forks source link

Specify brightness range when opening images in translucency mode #8905

Open bellzw opened 3 years ago

bellzw commented 3 years ago

Application version Cura 4.8.0

Platform Windows 10 64-bit running on a Lenovo Yoga 14 laptop

Printer BIBO Dual

Reproduction steps

  1. Set initial layer thickness 0.1 mm, layer height 0.1 mm
  2. Open png
  3. Set base = 0, height = 25.5, Transmission of 1 mm = 50%, Darker is higher, Color mode = Translucency, smoothing just above 0 (actually as low as I could make it and still see blue)
  4. Observe Actual/Expected results 1
  5. Clear work area
  6. Open png
  7. Set base = 1 mm, leave all other settings as they are
  8. Observe Actual/Expected results 2

Screenshot(s) Powers of 2 255 max - png containing squares with pixel values 128, 64, 32,16, 8, 4, 2, 1 against a background of 255.

slicing - result of Translucency mode import

1 mm base - result of setting base = 1 mm

Actual results

  1. Image is imported with the wrong height, 18.2 mm. The top of the tallest tower (pixel value = 1) is actually at slice 176, implying the height of the tower is 17.6 mm. The extra 0.6 mm appears to be some fins that protrude at the edges of the tallest tower. This appears to be an error in the smoothing algorithm. Fins are much worse if smoothing is turned off completely, however it is not clear that they are actually sliced and would print. I have not printed this because I fully expect the height to be a factor of 2.2 too large.

Tower steps are correctly linear because I stepped the pixel values by powers of 2.

  1. Image has the wrong height, 19.2 mm, but has the base correctly taken into account. In the 1 mm base screen shot, the fins on the tallest tower are very clear. The 176 slices for the towers remain, and are still incorrect.

Expected results

  1. Towers should not have fins at the edges, and the tallest tower should have been 8 mm (at slice 80) because the dynamic range is 255:1 corresponding to 8 thicknesses of 1 mm each attenuating by a factor of 2 (transmission of 1 mm was set to 50%).

  2. Towers should not have fins and model should be 9 mm high (1 for the base + 8 for the towers).

Project file BD_Powers of 2 255 max.zip

Log file

Additional information If the import is set up as Lighter is higher, instead of getting stepped holes, you will find that all the holes almost all go to the base.

Ghostkeeper commented 3 years ago

The problem is with the image in this case. I think it has been compressed with JPEG at some point. Here is a part of the darkest square (supposed to be pixel value 1) with the brightness extrapolated with a rather extreme gamma function:

image

Most of the square is value 1 indeed, but the top ridge is value 0 and there is some noise in the corner too. The ridge of 0 is being smoothed so it's not an extreme ridge in the final result, but it's there.

Due to the logarithmic function of the Translucency target, this one pixel value has a rather extreme effect on the model. Not really expected by the user perhaps, but the user did ask for a 100% opaque print so Cura tries as well as it can.

I think it's working as expected then.

bellzw commented 3 years ago

Ghostkeeper was exactly correct that the fins were a consequence of a jpg conversion at some time. I recreated the image with Paint and saved it as a png and the fins disappeared when I imported it with height = 25.5, base = 0, Translucency mode, darker is higher, transmission of 1 mm = 50%. Layer height is 0.1 mm and first layer is 0.1 mm.

The problem is not really the fins. It is the height of the pillars. Using the png with crisp edges, the height of the image is 17.6 mm, rather than the 8 mm it should be if 255 represents 100% transmission and 1 represents nearly (but not exactly) 0% transmission. It should be 8 mm because each mm of height means a factor of 2 reduction in transmission and the dynamic range of the image is 255:1, which is almost 2^8. The height is wrong by a factor of 2.2.

Log conversion - This is what Cura shows me when I import the image below with the previously stated settings.

powers of 2 - This image has crisp edges and none of the jpg noise.

The 3MF for this model is in the zip file. powers of 2.zip

Ghostkeeper commented 3 years ago

It should be 8 mm because each mm of height means a factor of 2 reduction in transmission and the dynamic range of the image is 255:1, which is almost 2^8. The height is wrong by a factor of 2.2.

That's not really how it works. By that logic, the "Height (mm)" parameter would have no effect on the model at all. But the formula by which it arrives at the height of each pixel is different as well.

The formula does a pre-conversion from perceptive brightness to actual transmittance, a screen gamma correction with a factor of 2.2 (which is coincidence; it has nothing to do with the 2.2x scale you're finding). It also applies a different factor to the red, green and blue channels in the image to get at the perceptive brightness (i.e. more weight to the green channel). It then re-maps the brightness to range from the transmittance of the base thickness to the minimum transmittance. It then maps that range to the logarithm of the desired brightness with as logarithmic base the transmittance at 1mm. And finally the alpha channel is multiplied in, if any.

As a result, the highest point in the model lands at a height of:

log(min_lum + (1-min_lum) * (brightness/255)^2.2)_transmit1mm

Where

That results in the following value:

log(0.5^25.5 + (1 - 0.5^25.5) * (1/255)^2.2)_0.5

Which is 17.5816. That's the height you're seeing in your model.

This is modelling the physical transmission of light, rescaled to the arbitrary desired range of transmission indicated by the base height and the total height in the pop-up. I think the formula is correct, although it's of course assuming a perfect uniform material which is not going to be correct if you have sparse infill or just because of air pockets and anisotropy.

bellzw commented 3 years ago

It took me a little time to organize my thoughts and GitHub doesn't seem to support the notation to which I am accustomed, so I had to prepare a pdf with my comments, analyses, and explanations. The bottom lines are that the factor of 2.2 comes from the gamma correction, allowing a user to supply Height (mm) is probably incorrect, and the formula for the highest point in the model appears to apply scaling to height_from_base prematurely.

beers law.pdf

Ghostkeeper commented 3 years ago

I consider the color->grey-scale conversion a separate process from the transmission calculation, so I use GIMP to generate grey-scale images for Cura. GIMP lets me select the factors for the color channels as I see fit and then I can adjust the dynamic range of the grey-scale image to cover [0, 255].

That's fine, of course. Conversion to greyscale is perceptual science and there are multiple ways to do it. Cura just uses one of the more common techniques. If you'd want to use another you'll need to provide the image as greyscale to Cura. I don't really see a reason for the user to customise this. These lithophanes are not really going to be an exact science, especially with how they're used in the field.

Oh, but if the factor of 2.2 is the same as the exponent 2.2 highlighted below, then it is exactly the factor I notice and not at all a coincidence!

It is kind of a coincidence. Since you landed on the 8mm height without using the Height (mm) setting which does affect my result. It's coincidence due to you specifying that the Height (mm) is 25.5mm. Any other value would result in a different factor. The actual difference in height (8mm vs 17.5816mm) is also 2.1977, not 2.2. Most lithophanes I've seen have a height difference more like 5mm. The exponential factor of 0.5^5 is more prominent then.

I will assume _transmit1mm means divide by transmit1mm, since that’s what the link to WolframAlpha shows me.

No, I intended it to mean the log base of transmit1mm. The Wolfram|Alpha link divides by log(transmit1mm) (with transmit1mm at 0.5). This is indeed a case where the Markdown of Github is not really sufficient, sorry.

I must admit though that from here on, it's not really appropriate for me to go into the remaining 2 pages of formulas from your analysis. I am not a physicist, and this time detracts from me working on more serious issues in Cura. I belayed my understanding of the source code as best I could, but it could be that I made a mistake somewhere. The implementation of the image loading was made by a material scientist and a 3D printing researcher at Ultimaker and I'm not completely at home there.

As far as I understand, the expected behaviour is that the perceptual brightness of the print (with a constant-brightness light source) should scale affinely with the perceptual brightness of the image on the user's screen. We're using the common "average" gamma value of 2.2 for that.

bellzw commented 3 years ago

I consider the color->grey-scale conversion a separate process from the transmission calculation, so I use GIMP to generate grey-scale images for Cura. GIMP lets me select the factors for the color channels as I see fit and then I can adjust the dynamic range of the grey-scale image to cover [0, 255].

That's fine, of course. Conversion to greyscale is perceptual science and there are multiple ways to do it. Cura just uses one of the more common techniques. If you'd want to use another you'll need to provide the image as greyscale to Cura. I don't really see a reason for the user to customise this. These lithophanes are not really going to be an exact science, especially with how they're used in the field.

We are in violent agreement. I was only pointing out that users have other options. I was not suggesting that the color-to-grey-scale be changed.

Oh, but if the factor of 2.2 is the same as the exponent 2.2 highlighted below, then it is exactly the factor I notice and not at all a coincidence!

It is kind of a coincidence. Since you landed on the 8mm height without using the Height (mm) setting which does affect my result. It's coincidence due to you specifying that the Height (mm) is 25.5mm. Any other value would result in a different factor. The actual difference in height (8mm vs 17.5816mm) is also 2.1977, not 2.2. Most lithophanes I've seen have a height difference more like 5mm. The exponential factor of 0.5^5 is more prominent then.

Cura produces the same result if I pick 50 mm for Height_mm. It really is the 2.2 in the exponent; it doesn't depend on the Height (mm) parameter. The small differences are a consequence of Cura scaling to the user-specified height within the logarithm argument, assuming you have relayed the correct formula. You will find Cura's result is even closer to 17.6 if you use 50 mm as the maximum height because 0.5^50 is even smaller than 0.5^25.5. I get 17.6 mm for the maximum height if I set that to 50 mm, or anything above 17.6 mm. If I set transmit1mm to 25 when converting the image, the result is 8.8 mm maximum thickness, which is also a factor of 2.2 off, because it should now take half has many layers. You will find the factor is even closer to 2.2 if you use 50 mm as the maximum height because 0.5^50 is even smaller than 0.5^25.5. Slicing my png (it has not been through the jpg process) with 1mm transmission of 70% results in a model that is 34.2 mm tall. I guarantee you that the factor of 2.2 comes from the exponent 2.2.

I will assume _transmit1mm means divide by transmit1mm, since that’s what the link to WolframAlpha shows me.

No, I intended it to mean the log base of transmit1mm. The Wolfram|Alpha link divides by log(transmit1mm) (with transmit1mm at 0.5). This is indeed a case where the Markdown of Github is not really sufficient, sorry.

I understand. Dividing by log(transmit1mm) is correct.

I must admit though that from here on, it's not really appropriate for me to go into the remaining 2 pages of formulas from your analysis. I am not a physicist, and this time detracts from me working on more serious issues in Cura. I belayed my understanding of the source code as best I could, but it could be that I made a mistake somewhere. The implementation of the image loading was made by a material scientist and a 3D printing researcher at Ultimaker and I'm not completely at home there.

As far as I understand, the expected behaviour is that the perceptual brightness of the print (with a constant-brightness light source) should scale affinely with the perceptual brightness of the image on the user's screen. We're using the common "average" gamma value of 2.2 for that.

Yes, once I saw the formula in your previous message, I understood the origin of the exponent and agree that it is appropriate if you're trying to duplicate the perceptual brightness from a display. One of my quibbles is whether a application of a gamma value is appropriate here since the light source is not a display. The other is the particular transformation to scale to the user-specified height implemented in Cura.

Perhaps it would be more useful to put me in touch with the person or persons who developed formulas? They might be able to set me straight, or I them. Or maybe it will turn into another feature in some later version.

Ghostkeeper commented 3 years ago

I don't suppose they're subscribed any more but sometimes tagging works: @BagelOrb

I do think that it would be appropriate to convert the brightness of the display to perceptual. After all, the lithophane doesn't have this transistor problem. It's just sunlight.

bellzw commented 3 years ago

I don't suppose they're subscribed any more but sometimes tagging works: @BagelOrb

Thank you for trying to involve @BagelOrb

I do think that it would be appropriate to convert the brightness of the display to perceptual. After all, the lithophane doesn't have this transistor problem. It's just sunlight.

I am not sure I understand the connection between your first and second sentences. I think you are saying that it is correct to drive a video display by applying voltage to a CRT or current to an LED according to brightness^2.2, but a lithophane's light source does have the same characteristics. You'll need to correct me if I've misunderstood.

A lithophane's light source, unlike a video display, is not driven by pixel values. A video display needs to be driven 2^2.2 harder to achieve the brightness implied by a doubling of a pixel value. (I wonder if video driver chips and cards do this in the hardware, or if they rely on the software to do this conversion.) However, the light source for a lithophane does not depend on the pixel value and the conversion needs only to account for the attenuation of light. This was one of my points in the pdf.

Let's hope @BagelOrb will join the conversation.

BagelOrb commented 3 years ago

I'm here! What's up?

The 2.2 exponent is used for signal compression, so it must be used to decode the rgb value to a light intensity value. There is a lot of confusion about this online.

bellzw commented 3 years ago

@BagelOrb: @Ghostkeeper and I have been discussing the reason that Cura generates a 17.6 mm model from an image with rectangles having values 128, 64, 32, ..., 2, 1 against a background of 255 when I specify the transmission of 1 mm to be 50% and the maximum height to be larger than 17.6 mm. If it were only modeling only the attenuation, the model should have been 8 mm tall because the dynamic range is 255:1. The reason is, indeed, the 2.2 exponent. @Ghostkeeper suggested the exponent was related to the gamma correction; you write that it is from decoding. He (she?) apparently hopes you can shed light (no pun intended) on the topic.

If it is a gamma correction, then, as I understand it, it is related to the eye’s response and the characteristics of displays. However, for a lithophane, the light source is not an LED display or electron beam hitting a phosphor, but a fixed luminosity bulb or sunlight. Consequently, to express the attenuation of an absorbing material, the 2.2 exponent does not seem correct, since the attenuation does not depend on the light source or the eye's response. My understanding is that the gamma correction is applied to a display so showing a brightness of 128 will appear half as bright as when it shows 255. LCDs and other displays have the correction built into the driving hardware.

If it indeed originates in the encoding/decoding of images, then I do not think any decoding of the rgb value necessary. I find that if I use Python Pillow functions to look at pixels in an image, I get the decoded values, not values^(1/2.2). I can create an image with Paint on my Windows laptop with known pixel rgb values, and get back the values I put in when I look at the file using image.getpixel(x,y) from PIL.

Perhaps the explanation I seek is whether the human eye interprets a factor of 2 attenuation of a bulb's light (half the cd/m^2) as a factor of 2 loss of brightness, or do we really see it as a factor of 0.5^2.2 (=0.218)? If the latter, I'd appreciate a reference.

We have also discussed the particular transformation used to account for the user's specified maximum height of the model, and we have differing opinions on how it should be done.

BagelOrb commented 3 years ago

This video explains it very clearly: https://youtu.be/LKnqECcg6Gw

The exponent isn't used for calibrating the display; it's for compressing the luminosity data to a format which is more suited to the human eye.

You can view a lithophane also as a display. The principle is the same: we need to convert from bytes (0-255) to relative luminance values (cd/m^2). Then those luminance values are converted into voltages for a screen and to thicknesses for a lithophane, but neither of these actually uses the 2.2 exponent.

You can verify whether those software packages work on decoded values or not by doing the blurring experiment shown by minutephysics.

bellzw commented 3 years ago

@BagelOrb ,thank you for the link. The minutephysics video is saying that it's our eyes that won't perceive a grey value of 128 as half of 255, but rather 21.8% of 255. Therefore in my 50% transmission in 1 mm example, the model really needs to be 2.2 times thicker than I expected.

I can verify that PILLOW retrieves the decoded pixel values from images. It returns 4 when I ask for a pixel that should be 4; it does not return 2 to my Python script. I also know that the electronics of displays include LUTs to account for the 2.2 exponent so the current/voltage is corrected to produce the correct number of cd/m^2. Also, GIMP passes the blurring test in the minutephysics video. I expect any software that decodes an image will receive the value of each pixel and will need the exponentiation to adjust it for the response of the human eye.

I'm still not sure it makes sense that the user can specify a maximum height, since the required thickness of plastic to guarantee the dynamic range in the image is determined by the opacity of the plastic, rather than the user's wishes. It seems that the height should be something determined by the properties of the image and plastic. If the user really wants to force the model to have a maximum height, then I think he/she needs to adjust the dynamic range of the image according to the opacity of the plastic.

Can you tell me if the formula provided by @Ghostkeeper, above, is how Cura converts an arbitrary brightness to height?

BagelOrb commented 3 years ago

The minutephysics video is saying that it's our eyes that won't perceive a grey value of 128 as half of 255, but rather 21.8% of 255.

No, they are saying that our eyes don't perceive half the number of photons as half the light intensity and therefore change the encoding such that 128 represents 21.8% of the photons compared to 255. This means that we do perceive 128 as half as grey as 255.

I'm still not sure it makes sense that the user can specify a maximum height

Perhaps some setting to specify the dynamic range instead makes more sense indeed.

Haven't double checked the formula @Ghostkeeper mentioned, but I trust he just obtained it from the code, so you wouldn't get a different answer from me.

BagelOrb commented 3 years ago

You're right that GIMP does blurring correctly. I guess you'd have to try blurring an image in python yourself to see...

bellzw commented 3 years ago

Yes, I misspoke (miswrote?). We need to have only 21.8% of the photons to "see" a half-scale brightness.

I will explore implications of the formula @Ghostkeeper provided and give some thought during the coming week of how to force a dynamic range. My understanding is that the human eye really only perceives 30 - 60 gradations of grey, so perhaps 5 or 6 bits of grey scale are sufficient. Probably, all of this is well plowed ground, but I'm retired and have the inclination.

It might be a nice feature if Cura showed the user the converted image prior to slicing.

bellzw commented 3 years ago

Cura Law.xlsx

I've finally found a big enough block of time to examine the ramifications of the conversion law @Ghostkeeper supplied on 22 Dec. You'll find 2 charts in which the transmission is calculated for various values of transmission1mm (I call it T1 for short) and height_from_base (H for short).

In T(P) for (T1)^H I calculate the transmission as a function of pixel value for a set of values of (T1)H, rather than H or T1 individually, because (T1)H is the combination that appears in the conversion law. The black curve is the transmission if Beer's law is used as I previously described, i.e., (T1)H = 0; a straight line on a log-log plot corresponding to pure (P/255)2.2. This is accomplished in real life when T1 is very small, or H is large, or the combination is very small. Even at 0.001, there is significant deviation from Beer's law for pixel values below ~30. I think that for decent results, a lithophane needs to span a dynamic range of at least 32:1, which means (T1)**H needs to be no greater than 0.031. For my filament with measured transmission1mm of 0.87, then I must specify H to be at least 24.9 mm to realize reasonable dynamic range.

So what happens if I specify a value of H that is smaller than what's required to get a minimum dynamic range?

In sheet T(H) for P, I calculate the set of transmission curves as a function of the height_from_base for a 32:1 dynamic range for a set of pixel values P spanning [0, 255]. H is expressed as H/H_min, with H_min the value that results in the specified dynamic range for the specified transmission1mm. The plot is on T(H) for P plot. It's a busy plot but shows the dynamic range of the lithophane given the user specified height relative to the minimum needed to achieve 32:1 dynamic range. You can change the value 32 in cell C2 of T(H) for P if you want to see the curves for different dynamic ranges. If a H is chosen larger than H_min, then the dynamic range will be better the whatever is in cell C2; I don't calculate for values beyond H/H_min =1. Using the example of my T1=0.87 filament, if I chose H=3.2 mm, a common value you find on YouTube videos, then H/H_min = 0.13 and the resulting dynamic range will be 1:0.64, ~1.56:1, and it will be much less than what is really needed. In T(P) for User H I show the transmission curve as a function of pixel value for H/H_min = 0.13 and T1 = 0.87. This plot is just a plot of a single row of T(H) for P, and you can see other rows corresponding to different H/H_min just by selecting a different row for the data plotted in the graph. Feel free to select log-log plotting when graphing rows close to H/H_min = 1.

The Dynamic Range sheet summarizes the ratio of the T(H) for P data by calculating and plotting the Dynamic Range computed as T(H) for P=255 divided by T(H) for P=1 and plotting as a function of H/H_min.

The bottom line is that selecting too small a thickness introduces a loss of dynamic range and decidedly non-logarithmic behavior.

I could do a similar analysis of the linear conversion, if I knew the formula. Does it map (P/255)**2.2 linearly onto [0,H]?

I'll make some Transparency mode and Linear mode lithophanes with a contrived image and try to measure transmissions to see if any of this analysis makes sense.

Ghostkeeper commented 3 years ago

I could do a similar analysis of the linear conversion, if I knew the formula. Does it map (P/255)**2.2 linearly onto [0,H]?

No, it doesn't apply gamma. It maps (P/255) on the range [base_height, base_height + H] linearly. It's meant to function more like a height map, which is a good application for linear I think. Predictable at least.

I think it's acceptable that having too small a thickness loses range. The lithophane is very thin then anyway, so you couldn't expect it to become very dark. If we can improve the formula though to get better perceptive relative brightness (closer to logarithmic), that would be nice. I really don't know how we should adjust the height calculation now though.

Need to keep in mind that all of this assumes a homogeneous interior, which is far from reality if the model is thicker than twice the top/bottom thickness (~2mm by default). It's advisable to use 100% infill when printing lithophanes, but not everyone does this. That is probably a much greater issue, and lower hanging fruit to solve.