Difference in Infragram & ImageSequencer ndvi-red results

[ccpandhare]

Original

Image Sequencer Output

Full Image

Infragram Output

Full Image

Is the algorithm we are using for ndvi-red correct?

    function changePixel(r, g, b, a) {
      var ndvi = 255 * (b - r) / (1.00 * b + r);
      return [ndvi, ndvi, ndvi, a];
    }

[jywarren]

Looks like negative values are getting output as positive in ImageSequencer, perhaps? See the black tree trunks? 🌲

Those are zero or negative values. Maybe negative values aren't getting zeroed out?

On Jul 1, 2017 12:26 PM, "Chinmay Pandhare" notifications@github.com wrote:

Original Thumbnail↓ Full Image Here https://preview.ibb.co/fx6TA5/Original.jpg

[image: Original] https://camo.githubusercontent.com/ad075f947c9d1277d44c6dffa7d6fc08db54a5e0/68747470733a2f2f7468756d622e6962622e636f2f634f6e44636b2f4f726967696e616c2e6a7067 Image Sequencer Output Full Image

[image: Image Sequencer] https://camo.githubusercontent.com/45a3453aacf7871c1306edbf9137020acd458acc/68747470733a2f2f707265766965772e6962622e636f2f6232356333512f496d6167655f53657175656e6365722e6a7067 Infragram Output Full Image

[image: Infragram] https://camo.githubusercontent.com/8a68d9157290278c7c2f3881a1611d1183ba44f5/68747470733a2f2f707265766965772e6962622e636f2f6575456e33512f496e6672616772616d2e6a7067

Is the algorithm we are using for ndvi-red correct?

function changePixel(r, g, b, a) {
  var ndvi = 255 * (b - r) / (1.00 * b + r);
  return [ndvi, ndvi, ndvi, a];
}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJyCyzvGFmmEBECfKyBdoZATE6fBKks5sJnMXgaJpZM4OLWyw .

[ccpandhare]

Changed the code to this:

    function changePixel(r, g, b, a) {
      var ndvi = 255 * (b - r) / (1.00 * b + r);
      ndvi = (ndvi>0)?ndvi:0;
      return [ndvi, ndvi, ndvi, a];
    }

Output:

I will check the infragram-js repository for details.

[ccpandhare]

I think Infragram uses the same algorithm.

ndvi = (nir, vis) ->
        n = nir.width * nir.height;
        d = new Float64Array(n);
        for i in [0...n]
                d[i] = (nir.data[i] - vis.data[i]) / (nir.data[i] + vis.data[i]);
return new JsImage(d, nir.width, nir.height, 1);

if mode == "ndvi"
       [r,g,b] = get_channels(img)
       ndvi_img = ndvi(r,b)
       # this isn't correct for NDVI; we want values from -1 to 1:
       # [[min],[max]] = ndvi_img.extrema()
       min = -1
       max = 1
       normalize = (x) -> (x - min) / (max - min)
       result = jsColorify(ndvi_img, (x) -> colormap(normalize(x)))
       update_colorbar(min, max)

Isn't it?

[jywarren]

Ah, i wonder -- do you need to be doing:

var ndvi = (255 / 2) * (1 + b - r) / (1.00 * b + r);

So that values from -1 to 1 are mapped from 0-255?

[ccpandhare]

I think NDVI should just be

var ndvi = (b - r) / (b + r)

Wikipedia

Update : Should It be the modulus of this?

[jywarren]

The issue is that there's a difference between what the NDVI value /is/ and what we can display in an image. Infragram.org decides that since there are negative values in NDVI, we should map actual NDVI values of -1 to 1, which are:

var ndvi = (b - r) / (b + r)

to the full range we can show in an image, which is 0 to 255, so, broken into two steps:

var ndvi = (b - r) / (b + r)
var pixel_value = 255 * (ndvi + 1) / 2;

[ccpandhare]

I am afraid that doesn't do the task either... Will look into this. This is the output when I used the above code:

[ccpandhare]

Somehow, We aren't reaching the extremes. I mean, The parts which are darker in the Infragram version are lighter here and the lighter ones are darker here.

[jywarren]

That's odd -- I ran this a few times manually with NDVI outputs:

ndvi = 1
> 1
255 * (ndvi + 1) / 2
> 255
ndvi = -1
> -1
255 * (ndvi + 1) / 2
> 0
ndvi = 0.33
> 0.33
255 * (ndvi + 1) / 2
> 169.57500000000002
ndvi = -0.33
> -0.33
255 * (ndvi + 1) / 2
> 85.425

This seems about right, no? Can you try outputting some of the pixel color values to be sure the NDVI is falling between -1 and 1? I don't really see how it could do otherwise.

[ccpandhare]

Yes, the ndvi value is between -1 and 1. Here's a PasteBin Link to the ndvi of each of the pixels of this image, for reference.

But interestingly, most ndvi values are close to zero.

[jywarren]

Can you output the R and B values? I wonder if they're not quite right... wrong scale or something? Maybe they're 0-100 for some reason?

[ccpandhare]

Sure, This PasteBin Link has r, b and ndvi values.

[jywarren]

Hmm, now that I think about it, isn't the last image you posted correct? That is, for areas of 0 NDVI, it should appear 50% grey. Infragram is rendering those black, which would be -1.

Do you want to post a question to the site on the topic and ask Chris Fastie and Ned Horning to confirm this?

https://publiclab.org/tag/ndvi has a lot of good info on this, and maybe test images others have used.

You could also ask a question there!

[ccpandhare]

Sure will do that! Thanks for the link!

[ccpandhare]

This is the link to the question I have asked on the website.

[ccpandhare]

I have some good news on this front, finally!

A Relief, Finally!

Chris Fastie suggested I try an image he gave. I think I got the right output from ImageSeqeucer. It even matched that of Infragram! But I do have Some interesting news here.

Original

Infragram Output

Image Sequencer Old Algorithm Output

This is the output of the algorithm you wrote earlier:

      var ndvi = 255 * (b - r) / (1.00 * b + r);
      return [ndvi, ndvi, ndvi, a];

Image Sequencer New Algorithm Output

This is the output using the newer algorithm:

      var ndvi = (b - r) / (b + r);
      var x = 255 * (ndvi+1) / 2;
      return [x, x, x, a];

What is surprising is that the older algorithm performed better. Whereas we had established that the newer algorithm maps all values. Moreover, Chris said that the new algorithm is correct, too!

So why didn't the older algorithm work for the previous image, is also another question.

[jywarren]

I think the older algorithm is flooring anything below zero! But you should post these results to Chris and ask him too:

https://publiclab.org/questions/ccpandhare/07-08-2017/how-to-verify-if-my-programmatically-generated-ndvi-version-of-an-image-is-correct

[ccpandhare]

Okay, Done that!

[ccpandhare]

Chris suggested another image. I tried that out. The same issue was there:

• Old algorithm : broken at negative values
• New Algorithm : faint.

So I created another algorithm "Improved Algorithm":

var ndvi = 255 * (b - r) / (b + r);
ndvi = (ndvi > 0) ? ndvi : 0;
return [ndvi,ndvi,ndvi,a];

This is basically the old algorithm, with the difference that I map negative values of ndvi to 0. This algorithm produced the exact same result as Infragram! I am not sure if this algorithm is correct!

Original Image

Infragram Output

Improved Algorithm

Old Algorithm

New Algorithm

[jywarren]

Ok, this is good to know. I think this may show how infragram.org is destroying some information when in non colorized mode. To do better we should preserve sub zero values and pass the whole thing to the colorized for easier viewing.

I think we're good here then!

On Jul 13, 2017 9:44 AM, "Chinmay Pandhare" notifications@github.com wrote:

Chris suggested another image. I tried that out. The same issue was there:

• Old algorithm : broken at negative values
• New Algorithm : faint.

So I created another algorithm "Improved Algorithm":

var ndvi = 255 * (b - r) / (b + r); ndvi = (ndvi > 0) ? ndvi : 0; return [ndvi,ndvi,ndvi,a];

This is basically the old algorithm, with the difference that I map negative values of ndvi to 0. This algorithm produced the exact same result as Infragram! I am not sure if this algorithm is correct! Original Image

[image: Original] https://camo.githubusercontent.com/bb5167aeacf98b25c1784076632c77f637b9d6cf/687474703a2f2f707265766965772e6962622e636f2f63376b7830462f76656761746174696f6e5f30322e6a7067 Infragram Output

[image: Infragram] https://camo.githubusercontent.com/45a2365a68e3b358eb94b5c54098d84e7d6e977a/687474703a2f2f707265766965772e6962622e636f2f6d73446a66462f696e6672616772616d2e706e67 Improved Algorithm

[image: Improved Algorithm] https://camo.githubusercontent.com/123c95539c1499ec7f7de8fbfc5cc07eeaa97686/687474703a2f2f707265766965772e6962622e636f2f6834703153762f696d70726f7665642e6a7067 Old Algorithm

[image: Old Algorithm] https://camo.githubusercontent.com/d6118cbc5b4317d1ce659ea22d401692e0675273/687474703a2f2f707265766965772e6962622e636f2f6e4445714c462f6f6c642e6a7067 New Algorithm

[image: New Algorithm] https://camo.githubusercontent.com/da1fb28e0c2565519c64c48ec392605a364d012f/687474703a2f2f707265766965772e6962622e636f2f6651333675612f6e65772e6a7067

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-315081822, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ7DWMh90gT-uPdOaUneeCwrT4NlKks5sNh84gaJpZM4OLWyw .

[ccpandhare]

This is definitely not the right method. It worked for this:

But not for this:

Interestingly, mapping all negatives to 0 and all positives to 255 worked for the second one.

So are we missing something in the algorithm?

Maybe our mapping isn't correct. Maybe we shouldn't have a linear map at all. Maybe we should have a linear map for a certain range of ndvi values and them map values exceeding them to 0 or 255. Something like:

-1 to -x : 0
-x to 0 : linear function
0 to x : linear function
x to 1 : 255

What do you think about this? This will solve all test cases. But I am afraid this isn't right.

On the other hand: I totally agree with this:

I think this may show how infragram.org is destroying some information when in non colorized mode.

This is certainly the correct explanation.

[ccpandhare]

I think there is data rounding off in Infragram as you pointed out.

[jywarren]

Yes. I think the lower contrast image is actually the correct one. To make it more readable we should colormap it.

On Jul 13, 2017 10:04 AM, "Chinmay Pandhare" notifications@github.com wrote:

I think there is data rounding off in Infragram as you pointed out.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-315087341, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ7cR7clPMqhC-ExCzoUD6epGRCIgks5sNiOvgaJpZM4OLWyw .

[Fastie]

Jeff might be right that the lower contrast version ("new algorithm") is correct. The NDVI values for that photo are mostly between -0.2 and +0.3, so there should be very little black or white in the grayscale image, assuming that -1 is black and +1 is white. Here is the histogram of NDVI values computed by Fiji:

However, Ned's Photo Monitoring plugin produces an image with high contrast just like the "Infragram" or "Improved Algorithm" versions above. So Infragram and the Photo Monitoring plugin might be computing NDVI correctly but mapping the grayscales strangely. Ned could help with the mapping done by the plugin.

Below is the histogram of NDVI values for Clayton's Chicka Boom row crop photo with all NDVI values > 0.

[jywarren]

very helpful. i think we're on the right track then, and part of the NDVI module may need to be a config option to set the output range (defaulting to -1=>1 mapping to 0-255).

Also, i think we should make a histogram module! Kinda weird, but it could output a histogram graph given an image input! but let's think about that later (it could use plotly, though inefficiently, and would probably break some module rules)

On Thu, Jul 13, 2017 at 10:44 AM, Fastie notifications@github.com wrote:

Jeff might be right that the lower contrast version ("new algorithm") is correct. The NDVI values for that photo are mostly between -0.2 and +0.3, so there should be very little black or white in the grayscale image, assuming that -1 is black and +1 is white. Here is the histogram of NDVI values computed by Fiji: [image: w25floathist] https://user-images.githubusercontent.com/1805899/28171468-82a3968c-67b6-11e7-8eac-9df84e1abd33.PNG

However, Ned's Photo Monitoring plugin produces an image with high contrast just like the "Infragram" or "Improved Algorithm" versions above. So Infragram and the Photo Monitoring plugin might be computing NDVI correctly but mapping the grayscales strangely. Ned could help with the mapping done by the plugin.

Below is the histogram of NDVI values for Clayton's Chicka Boom row crop photo with all NDVI values > 0.

[image: claytonndvihist] https://user-images.githubusercontent.com/1805899/28171870-d37ff586-67b7-11e7-8654-fb08a66de40d.PNG

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-315099307, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ0hzHcv-zMfwsZUtSUJ7OdDf1QUQks5sNi1PgaJpZM4OLWyw .

[ccpandhare]

I will open up an issue for the histogram module. Also, What colormap should we use then? Or should we use one at all?

[jywarren]

I'm not sure -- isn't a basic histogram just a graph, not colorized? Maybe stick with the simplest version for now?

[ccpandhare]

Yes, true. I was actually referring to the colormap we were going to apply to ndvi-red in order to get better contrast for the viewers...

[jywarren]

Oh haha ok yeah. So last I remembered we broke colormap into its own module. Are we talking about what the default is or what we should show in the example demonstrating ndvi? Again, apologies if I'm slow at the moment.

On Jul 26, 2017 7:00 AM, "Chinmay Pandhare" notifications@github.com wrote:

Yes, true. I was actually referring to the colormap we were going to apply to ndvi-red in order to get better contrast for the viewers...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-317948019, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ9DxxsKUfHAgccIQml2876w6oEADks5sRsf-gaJpZM4OLWyw .

[ccpandhare]

No no, I totally understand that you have a lot to look after!

Actually we were getting different outputs for ndvi-red than what Infragram gives. After a lot of tests, we concluded that Image Sequencer gave the right results, and that Infragram uses some "colormap" to enhance the image for better viewing.

The histogram thing was completely different, Sorry. That must have confused you - I shouldn't have said everything in one comment.

[jywarren]

But didn't I see a separate colormap module you wrote? I think we can just consider ndvi separately from color mapping; after all, Infragram does them as two separate steps. Does that answer your question?

On Jul 26, 2017 9:19 AM, "Chinmay Pandhare" notifications@github.com wrote:

No no, I totally understand that you have a lot to look after!

Actually we were getting different outputs for ndvi-red than what Infragram gives. After a lot of tests, we concluded that Image Sequencer gave the right results, and that Infragram uses some "colormap" to enhance the image for better viewing.

The histogram thing was completely different, Sorry. That must have confused you - I shouldn't have said everything in one comment.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-317970340, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ7XgX0QdDBVF4QnpC0AS68ytW0iEks5sRuiWgaJpZM4OLWyw .

[ccpandhare]

Yes there is a separate colormap module. Yes, this does solve my question. I was under the impression that we should have a colormap too. Okay then, I think this can be closed now and ndvi-red is good to go!

[jywarren]

Wait, do we need an NDVI module which accepts an option for which channels to use, default to red/blue? Rather than this? Or maybe ndvi-red is just a wrapper if ndvi which defaults to red? Or am I overthinking this.

On Jul 26, 2017 10:22 AM, "Chinmay Pandhare" notifications@github.com wrote:

Yes there is a separate colormap module. Yes, this does solve my question. I was under the impression that we should have a colormap too. Okay then, I think this can be closed now and ndvi-red is good to go!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/34#issuecomment-317984152, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJyAiC5faM_POAiAQmrM1plUI_M5bks5sRvcpgaJpZM4OLWyw .

[ccpandhare]

It could be renamed to ndvi and default to R and B channels. As of now it doesn't accept any such parameters.