Question about your notebook "Adaptive Contouring in Fragment Shaders"

[neozhaoliang]

@rreusser

Hi Ricky: I really enjoy the notebooks you have shared on Observablehq. They are beautifully designed and your explanations are fantastic. Thank you for generously sharing your knowledge!

Recently, I have been studying your notebook Locally Scaled Domain Coloring Part 1: Contour Plots. However, there are some parts that I don't quite understand, so I was hoping to ask for your help.

I know that opening an issue to ask questions in your personal project may not be the most appropriate approach, but mathstodon has a limitation on the words I can send, and your twitter account seems has been deleted. If that is indeed the case, please allow me to extend my apologies to you.

Below are my questions:

The first part that I'm having trouble with is the following:

I can understand that both sides are using values in pixels, but I don't quite understand why it can be used to draw contours with uniform width. I understand that for a function f, (0.5-abs(fract(f)-0.5)) gives the distance between a pixel and contour lines $\{ f(x)=k\mid k\in\mathbb{Z} \}$. But it seems difficult to predict the change of (0.5-abs(fract(f) - 0.5))/fwidth(f) between different pixels? (Did I miss something?) So I don't quite understand why doing it as below can give contours with uniform width.

Another aspect that I don't quite understand is how the value of N_octave below is derived:

It seems in the code below this paragraph, $\nabla f$ is calculated using screen-space derivatives, while $f$ is a standard unit. I don't quite understand why this is being done. I apologize if these questions were explained in your article or if they are self-evident. My level of expertise in shaders is limited, and I have been unable to understand it for some time. So I hope you can provide some help and answer my questions. I'm sorry to bother you and thank you very much!

[rreusser]

No, this is great! And apologies it's a bit awkward to contact me ever since "the twitter thing," but this is a perfectly fine place! 🙇

The logic is definitely tricky to follow, and I won't assert it's necessarily the best or cleanest formulation. A fundamental assumption in the whole thing is linear on the scale of a pixel. I'll give just an ever so slightly different formulation which might be clearer, and I don't know, might even be better:

1. consider y = 3.5 sin(x).
2. Now plot mod(f(x) - 0.5, 1) - 0.5. You now have a function that cross y=0 every time f(x) = n for some integer n. Of course the slope at these crossings is whatever the function's slope is.
3. So now divide by df/dx. If we linearize a function as it crosses zero as y = m x, then the slope of the linearization is m. Dividing by the slope of the function, which at this crossing is m, we have for the linearization y = x. So the function has value zero and slope 1 every time the original function crosses an integer:
4. If we take the absolute value, apply a constant scale factor to account for the scale of x relative to pixel widths (so basically the chain rule, d/dpixel = d/dx * dx/dpixel), then we can threshold the function. If we do this correctly, then we have a function that goes from 0 to 1 over precisely one pixel width.
5. If we threshold this function (< 1: black, >= 1: white) then we have single-pixel-width contours of the function.

I think this is slightly different than the function I used, but with the possible exception of some details I haven't totally explored like how it handles just touching contour lines or something, I believe it's equivalent.

Let me know if that helps regarding the first part. Taking a look at the second…

[neozhaoliang]

Wow, these explanations are patient and meticulous, which is really great! Thank you very much for taking the time to prepare the illustrations. I think I can understand the first part now.

[rreusser]

The second part is more tricky. It's been a while, but it looks like I was plotting contours of the the log of the function using the method above. We now know the contouring part; the question here is the contour spacing. We can't use a single contour spacing since they'll be too dense some places and too sparse in others. To decide the contour spacing, I've just used the derivative of what we're plotting. Flat sections get more densely spaced contours so that we see something even when there's not much change, while steep sections have more spaced out contours.

So let's start with the gradient of what we're contouring. We're contouring log(f). The magnitude of the gradient is grad( |log(f)| ), but by the logarithmic derivative, this is easier to compute as just the derivative divided by the function, |grad( f )| / |f|.

In terms of GLSL, the gradient of the log corresponds to:

// |grad(log2(abs(f)))|:
hypot(vec2(dFdx(log2(abs(f))), dFdy(log2(abs(f)))))

While an equivalent and nicer form is:

// |grad(f)| / abs(f):
hypot(vec2(dFdx(f), dFdy(f))) / abs(f)

This is the gradient which describes "how dense are contours", except this function changes continuously. Our contour spacing can't change continuously. We could take the floor (or ceil) to turn this into regions of constant spacing, but since we're plotting the log of the function, what turns out to work is to take the ceil in log space. That is, we write grad = exp( log( grad ) ) and then throw ceil into the middle of it, gradRegions = exp( ceil( log( grad) ) ).

This log-ceil-exp transform looks like the following:

Interestingly, it's scale-independent. As we zoom in, the flatting into constant regions looks exactly the same! This the key property that gets us what we want. Applying this to the gradient, we have the gradient which shows how fast the log of the function is changing, except now it also is flattened into regions of constant value in a manner that works at any scale.

So the equation you asked about is sort of half of this, written honestly not very clearly:

octave = ceil( log( grad(log2(|x|)) ) )

except I wrote this as

octave = ceil( log( grad(f) / |f| ) )

and threw in a constant scale factor m to account for the fact that GLSL gives us a screen space gradient. I also took the log in base D. If D = 2 each successive octave splits into two divisions at the next octave. If D = 3, then it splits into 3. Raising it back to the power of D^octave, we just have our spacing.

I think that gets us about to the end. Once we have the contour spacing, the procedure above applies contours. If we iterate over a couple adjacent octaves and apply a bit of blending using the contour spacing without the ceil function, then we get the octaves to fade in and out smoothly, which makes it look a bit more seamless.

Reading back, it took me a while to recall how this works. Apologies it wasn't written more clearly the first time around, but I hope the above clears some things up. Please let me know if any of this is still unclear!

[neozhaoliang]

These past few days, I have been studying your answers and trying my best to understand them on my own to avoid constantly bothering you with questions. Your answers are always so detailed that it makes me feel embarrassed. However, I still feel like I am lacking an intuitive understanding of how to go from mathematical derivation to displaying shaders. Below, I am attempting to rephrase your answer based on my understanding, and I was hoping you could check if I have correctly understood your meaning.

For the 1D case, as you have explained, the ratio f/df behaves similarly to the line y=x near f=0. If we use the screen-space derivative dF instead of df (I used dF to denote the screen-space derivative, to distinguish it from the usual derivative df), and compensate with the scaling factor dx/dpixel, then the variable x of f/dF is still in the usual units, but the value (y-coordinate) is in pixels. The expression g = 0.5 - (abs(fract(f)-0.5)) represents the distance (in units) between x and the contour, but luckily since we have dg = df. Therefore, dG = dF. Hence g/dF also behaves like y=x near g=0 (and g=0 is exactly the contour line!). So when g/dF belongs to [0,1], that would mean the distance between the pixel corresponding to x and the contour is less than 1 pixel. This allows us to draw a contour with a width of 1 pixel.

Is my understanding correct?

[neozhaoliang]

In the case of 2D, I can understand the step of computing grad(log|f|), which can be represented as grad(f)/ abs(f). We can continue to estimate grad(f) using screen space derivatives and then multiply it by the ratio factor between coordinates and pixels.

Now let g = |grad(f)| / abs(f), so g is the absolute value of the derivative of log f. You used octave=ceil(log(g)) (in base D) to determine which octave x belongs to. I didn't understand the difference between using ceil and floor. They should only differ by a shift of one octave, right? Does that mean that using the floor function can cause numerical errors for certain values?

I can understand that taking the logarithm of the gradient g has the effect of smoothing out the slope of the curve. When g is large, it means that the contour lines of logf are very dense, and taking the logarithm can ensure that this part of the area falls within the same octave as much as possible. When g is small, that is, we are in a flat area of logf, the octaves will also be divided finely to obtain as many contours as possible.

[rreusser]

Your answers are always so detailed that it makes me feel embarrassed.

Please don't think of it that way! I spent a chunk of time just staring at my own explanations from a year ago, shaking my head and struggling to understand what I was trying to say and how the thing works. It's complicated, and I don't think I've done a particularly clear job of explaining it!

Is my understanding correct?

Yes, I think so. Here's a really pared-down shadertoy you can experiment with: https://www.shadertoy.com/view/md3GRN (Note that this shadertoy plots contours of integer-spaced values, while my fancy shading thing uses the log of the function as its input. If you took the log of some function right on the first line though, then the rest of it would work exactly the same, without modification. It would just instead be contours of the log of the function.)

I think your explanation is right, so to try to really make it concrete, here are some numbers: You're making a plot with scale dx/dpixel = 0.1, or in other words, it x increase from 0 to 1 over ten pixels.

Now plot a function with slope df/dx = 1. The scale of the drawing means the function increases from 0 to 1 over ten pixels. If we divide by dx/dpixel, then df/dpixel = df/dx * (dx/dpixel) = 0.1. This means that y increase by 0.1 for every pixel. This is the value that dFdx of OES_standard_derivatives computes.

So given that the value we start with when we compute dFdx(f) is df/dpixel = 0.1, meaning it goes from 0 to 0.1 over a single pixel, then if we divide by dx/dpixel = 0.1, then we have a function which is 0 at the zero of the function and goes to 1 a single pixel away.

They should only differ by a shift of one octave, right?

Correct, floor is absolutely as valid as ceil, and there's no difference in numerical error or anything. My numbers just happened to work out such that, relative to the smallest contour spacing, the shift of +1 due to ceil instead of floor gave me a scale that matched my expectations.

[neozhaoliang]

Thank you for your shadertoy example. It really helped me understand the concept better. By the way, does the code's "abs(fract(f) - 0.5)" draw the value of f at half-integer points?

I think I now understand your second part, on how to draw contours with uniform spacing. A simple way to think about it is to imagine a ruler with tick marks placed at $D^N$, where $N\in\mathbb{Z}$. These marks divide the ruler into intervals like $(D^{N-1},D^N]$. For g=|grad(f)/f|, we find the interval or "octave" it falls into, which is N if one uses ceil, and N-1 if floor is used. Then we draw the contours located at $|\log f|=k*D^N,k=1,2,\ldots, D-1$ in that octave (these contours actually lie in the interval $(D^N, D^{N+1}]$ in terms of log f values), where the screen-space derivative of log f / D^N is the 1 / widthscale in your code.

Update: I have finished reading the entire notebook, and I believe I can understand the remaining content. This issue can be closed now. Thank you very much for your patient assistance once again.

[rreusser]

Oh, thank you! That should have been abs(fract(x-0.5) - 0.5), which looks like this:

And yes, that's exactly it. Things sit in intervals, and for reasons I didn't really pursue, rounding up was better than rounding down. 🤷

[rreusser]

Awesome, I'm glad to hear it, @neozhaoliang ! Apologies it didn't just explain itself right from the start. I hope it provides something of value though, and if you make use of it somehow, I'd love to hear about it!

Parting thoughts:

• the standard derivatives trick means this works just the same on triangle meshes as well; it doesn't have to be a flat plane.
• I've wanted to make use of this to make proper plotting tools for lines, implicit, complex functions, surfaces, etc. But I just haven't had the time. 🤷 thetamath.com is a big inspiration, but would need to be built out more (and IMO in a more common language) to really be useful. Desmos is amazing, but I think there's a lot more possibility there. I want something with sort of the possibility of shadertoy, but basic primitives like lines, points, equations, text labels, and mouse/touch interaction so that you can use it to put together ideas way faster.
• I'm sure a lot is lost in google translate, but your website has lots of interesting content!

[neozhaoliang]

the standard derivatives trick means this works just the same on triangle meshes as well; it doesn't have to be a flat plane.

Got that. Actually, this is the first time I learned about the chain rule for screen-space derivatives and coordinate derivatives, and the divide by gradient trick. They seem so obvious, but I have never come across such a clear explanation like yours when I searched on Google before. It has been very helpful to me.

... I want something with sort of the possibility of shadertoy, but basic primitives like lines, points, equations, text labels, and mouse/touch interaction so that you can use it to put together ideas way faster.

Your observable notebooks area already awesome enough. Usually I go to https://graphtoy.com/ by iq for quick 1D function plots, and matplotlib for more complex figures (Regarding JS, I am a layman). If you ever start developing a website like you mentioned, I would be happy to help contribute code && improve the user experience.

My main goal in studying your work is to use it to create visualizations of complex functions. I'm not sure if you're familiar with the books Visual Complex Analysis by Needham and Indra's Pearls by Mumford, both of which are great mathematical readings with many interesting illustrations. My next plan is to use ShaderToy to implement some interesting content from these two books. Your methods have been very helpful to me.

I'm sure a lot is lost in google translate, but your website has lots of interesting content!

Thank you for the compliment. I usually take notes and document my interesting projects in Chinese. You can find all the source code on GitHub and ShaderToy, so nothing lost in the sense of code :P. In fact, I'm also using ChatGPT to help me polish my writing to communicate with you. Although I can write in English directly, I'm always afraid of using improper wording that could cause misunderstandings. :grin:

[neozhaoliang]

@rreusser Hi Ricky: I recently came across your stunning animations on line integral convolution. Since you have made several animations on topics in complex analysis, I want to reach out and ask whether you might be interested in creating more content like this. I'm asking this because I'm considering adding some visual animations to Needham's "Visual Complex Analysis" book, which offers ideal material for making Observable notebooks and I believe aligns with your expertise.

I have created some animations in this area before (Mobius transformations, Marden's theorem, Rouche's theorem, all these can be found in Needham's book), but they were quite rough, and I wasn't satisfied with their quality. My primary development language is Python, and creating notebooks as beautifully as you do with JavaScript is challenging for me. And there are lots of more topics to be explored in Needham's book. So, please allow me to politely ask (I hope you won't mind) if you would be willing to consider creating such a series? I can provide assistance as possible as I can, or if you prefer, we could collaborate on the same repository or notebook.

I hope my request doesn't come across as offensive, and I wish you all the best in creating more brilliant work in the future.

[rreusser]

@neozhaoliang Apologies I didn't respond to the above sooner! Due to factors beyond my control, I've been very distracted for the past two months and have barely been able to get regular work done, much less work on visualizations.

I would absolutely love to produce interesting content in this area (also, in my opinion, the quality of your work on shadertoy is stunning! 😍). I've wanted to make a graphing calculator tool for a very long time, but I've always gotten stuck trying to do too much rather than keeping it simple.

Needham's book looks really interesting. What do you have in mind for the format and audience of this sort of content? One thing I'm uncertain of is whether the goal is to produce standalone visualizations (this part is relatively easy) or general tools which can be used to produce visualizations (I find this part relatively difficult).

The tool I dream of is something sort of like desmos in that it facilitates mathematical visualization, but with a feel closer to shadertoy so that you can access pretty low level things like shaders or geometric drawing primitives.

Anyway, just let me know what you have on your mind! Hopefully after the end of November, my mind will be a little more free to do some coding for fun.

[neozhaoliang]

Apologies I didn't respond to the above sooner! ...

There's absolutely no need to apologize for the response time. In my experience, to code fun programs one must be in a relaxed and enjoyable mode. It feels quite different when there's a sense of urgency, so I genuinely hope I haven't made you feel pushed at all.

I would absolutely love to produce interesting content in this area (also, in my opinion, the quality of your work on shadertoy is stunning!

So glad you love this, and thank you for your praise!

What do you have in mind for the format and audience of this sort of content?

In my opinion, I hope the audience for this project will be students, engineers learning complex analysis, or teachers in this field searching for content for their class, to assist them in appreciating the beauty of complex analysis more intuitively and developing a fondness for it.

There are certainly various suitable options for presenting the content. As of now, maybe a series of Observablehq notebooks with each notebook discussing a specific topic/theorem would be a good choice? You are an expert in this realm, so, I assume for you this would be the quickest (or most comfortable) way to get started. Of course, I'm a newcomer to Observablehq, but I'm eager to learn how to use it.

One thing I'm uncertain of is whether the goal is to produce standalone visualizations (this part is relatively easy) or general tools that can be used to produce visualizations (I find this part relatively difficult).

I'm not clear how "general" the tools would be. I see you have written some libs like regl, regl-gpu-lines, they are very helpful and are on my list to learn. Do you mean features like react hooks, or rending math formulas as texts in webgl-canvas?

The tool I dream of is something sort of like desmos in that it facilitates mathematical visualization, but with a feel closer to shadertoy so that you can access pretty low-level things like shaders or geometric drawing primitives.

That sounds cool! If there's a website like that, I think we could have a community similar to Shadertoy but composed of math people! If you have a prototype, I'd be happy to try it out and contribute!