Closed wpalfi closed 3 years ago
This is an interesting observation! All I can think of is this: Linear interpolation creates more complex relationships between texel values and pixel colors, and the resulting optimization problem becomes somehow badly conditioned.
As each lookup is a blend from several texel values, there are multiple ways to approach the right answer, and this is a tricky situation to optimize. SGD clearly doesn't work well in this situation, but Adam reaches high accuracy quite rapidly with learning rates such as 1e-3 or 1e-2. The loss doesn't go all the way to zero, but this isn't very surprising because the computations are done in 32-bit floating point for performance reasons which limits the precision of the gradients.
The following script optimizes the texture of two triangles towards zero.
I get the expected result when I run it with![image](https://user-images.githubusercontent.com/18048248/124949325-c0e17900-e011-11eb-85a9-798bb74a59fe.png)
filter_mode='nearest'
:But when I switch to![image](https://user-images.githubusercontent.com/18048248/124947774-6dbaf680-e010-11eb-85af-369a9033800c.png)
filter_mode='linear'
, the training of the pixels at the edges slows down after ~100 iterations and pixel values are almost stuck at small constant values.The plot looks similar with mipmapping enabled.