Something wrong with MOD09 250m lat/lons

[captainkirk99]

These don't look right:

 Latitude_250m =
  81.8725967407227, 81.8725967407227, 81.8725967407227, 81.8725967407227, 
    81.8851928710938, 81.8851928710938, 81.8851928710938, 81.8851928710938, 
    81.8974685668945, 81.8974685668945, 81.8974685668945, 81.8974685668945, 
    81.9094390869141, 81.9094390869141, 81.9094390869141, 81.9094390869141, 
    81.9211044311523, 81.9211044311523, 81.9211044311523, 81.9211044311523, 
    81.9324722290039, 81.9324722290039, 81.9324722290039, 81.9324722290039, 
    81.9435424804688, 81.9435424804688, 81.9435424804688, 81.9435424804688, 
    81.9543228149414, 81.9543228149414, 81.9543228149414, 81.9543228149414, 
    81.9648208618164, 81.9648208618164, 81.9648208618164, 81.9648208618164, 
    81.9750366210938, 81.9750366210938, 81.9750366210938, 81.9750366210938, 
    81.984977722168, 81.984977722168, 81.984977722168, 81.984977722168, 
    81.9946594238281, 81.9946594238281, 81.9946594238281, 81.9946594238281, 
    82.0040664672852, 82.0040664672852, 82.0040664672852, 82.0040664672852, 
    82.0132141113281, 82.0132141113281, 82.0132141113281, 82.0132141113281, 
    82.0221099853516, 82.0221099853516, 82.0221099853516, 82.0221099853516, 
    82.0307464599609, 82.0307464599609, 82.0307464599609, 82.0307464599609, 
    82.0391464233398, 82.0391464233398, 82.0391464233398, 82.0391464233398, 
    82.0472946166992, 82.0472946166992, 82.0472946166992, 82.0472946166992, 
    82.0552062988281, 82.0552062988281, 82.0552062988281, 82.0552062988281, 
    82.0628814697266, 82.0628814697266, 82.0628814697266, 82.0628814697266, 
    82.0703277587891, 82.0703277587891, 82.0703277587891, 82.0703277587891, 
    82.0775527954102, 82.0775527954102, 82.0775527954102, 82.0775527954102, 
    82.0845489501953, 82.0845489501953, 82.0845489501953, 82.0845489501953, 
    82.0913238525391, 82.0913238525391, 82.0913238525391, 82.0913238525391, 
    82.0978851318359, 82.0978851318359, 82.0978851318359, 82.0978851318359, 
    82.1042327880859, 82.1042327880859, 82.1042327880859, 82.1042327880859, 
    82.1103744506836, 82.1103744506836, 82.1103744506836, 82.1103744506836, 
    82.1163101196289, 82.1163101196289, 82.1163101196289, 82.1163101196289, 
    82.1220474243164, 82.1220474243164, 82.1220474243164, 82.1220474243164, 
    82.1275787353516, 82.1275787353516, 82.1275787353516, 82.1275787353516, 
    82.1329193115234, 82.1329193115234, 82.1329193115234, 82.1329193115234, 
    82.138069152832, 82.138069152832, 82.138069152832, 82.138069152832, 
    82.1430358886719, 82.1430358886719, 82.1430358886719, 82.1430358886719, 
    82.1478118896484, 82.1478118896484, 82.1478118896484, 82.1478118896484, 
    82.1524047851562, 82.1524047851562, 82.1524047851562, 82.1524047851562, 
    82.1568145751953, 82.1568145751953, 82.1568145751953, 82.1568145751953, 
    82.1610488891602, 82.1610488891602, 82.1610488891602, 82.1610488891602, 
    82.1651153564453, 82.1651153564453, 82.1651153564453, 82.1651153564453, 
    82.1690063476562, 82.1690063476562, 82.1690063476562, 82.1690063476562, 
    82.172737121582, 82.172737121582, 82.172737121582, 82.172737121582, 
    82.1762924194336, 82.1762924194336, 82.1762924194336, 82.1762924194336, 
    82.1796951293945, 82.1796951293945, 82.1796951293945, 82.1796951293945, 
    82.1829299926758, 82.1829299926758, 82.1829299926758, 82.1829299926758, 
    82.1860122680664, 82.1860122680664, 82.1860122680664, 82.1860122680664, 
    82.1889343261719, 82.1889343261719, 82.1889343261719, 82.1889343261719, 
    82.1917114257812, 82.1917114257812, 82.1917114257812, 82.1917114257812, 
    82.1943359375, 82.1943359375, 82.1943359375, 82.1943359375, 
    82.1968154907227, 82.1968154907227, 82.1968154907227, 82.1968154907227, 
    82.1991424560547, 82.1991424560547, 82.1991424560547, 82.1991424560547, 
    82.2013320922852, 82.2013320922852, 82.2013320922852, 82.2013320922852, 
    82.2033843994141, 82.2033843994141, 82.2033843994141, 82.2033843994141, 

Here's the 500m ones:

 Latitude_500m =
  81.8725967407227, 81.8725967407227, 81.8851928710938, 81.8851928710938, 
    81.8974685668945, 81.8974685668945, 81.9094390869141, 81.9094390869141, 
    81.9211044311523, 81.9211044311523, 81.9324722290039, 81.9324722290039, 
    81.9435424804688, 81.9435424804688, 81.9543228149414, 81.9543228149414, 
    81.9648208618164, 81.9648208618164, 81.9750366210938, 81.9750366210938, 
    81.984977722168, 81.984977722168, 81.9946594238281, 81.9946594238281, 
    82.0040664672852, 82.0040664672852, 82.0132141113281, 82.0132141113281, 
    82.0221099853516, 82.0221099853516, 82.0307464599609, 82.0307464599609, 
    82.0391464233398, 82.0391464233398, 82.0472946166992, 82.0472946166992, 
    82.0552062988281, 82.0552062988281, 82.0628814697266, 82.0628814697266, 
    82.0703277587891, 82.0703277587891, 82.0775527954102, 82.0775527954102, 
    82.0845489501953, 82.0845489501953, 82.0913238525391, 82.0913238525391, 
    82.0978851318359, 82.0978851318359, 82.1042327880859, 82.1042327880859, 
    82.1103744506836, 82.1103744506836, 82.1163101196289, 82.1163101196289, 
    82.1220474243164, 82.1220474243164, 82.1275787353516, 82.1275787353516, 
    82.1329193115234, 82.1329193115234, 82.138069152832, 82.138069152832, 
    82.1430358886719, 82.1430358886719, 82.1478118896484, 82.1478118896484, 
    82.1524047851562, 82.1524047851562, 82.1568145751953, 82.1568145751953, 
    82.1610488891602, 82.1610488891602, 82.1651153564453, 82.1651153564453, 
    82.1690063476562, 82.1690063476562, 82.172737121582, 82.172737121582, 
    82.1762924194336, 82.1762924194336, 82.1796951293945, 82.1796951293945, 
    82.1829299926758, 82.1829299926758, 82.1860122680664, 82.1860122680664, 
    82.1889343261719, 82.1889343261719, 82.1917114257812, 82.1917114257812, 
    82.1943359375, 82.1943359375, 82.1968154907227, 82.1968154907227, 
    82.1991424560547, 82.1991424560547, 82.2013320922852, 82.2013320922852, 

[captainkirk99]

OK, as explained in this issue: https://github.com/SpatioTemporal/STAREmaster/issues/42

This is not broken, but by design.

The lat/lon values are provided only at the 1 km resolution. So for the 250 m data, we just use the same 1 km resolution lat/lon values 4 times. This was suggested by the MODIS support team at NASA.

Another alternative would be to interpolate between them, but this was decided to not be appropriate at a STARE meeting.

As a result, the 250 m values are going to just be the 1 km values, repeated 4 times. That doesn't seem like a good answer. I'm very willing to try something else, including a linear interpolation between the 1 km values, which, although not entirely accurate, due to the spherical geometry of the Earth's surface, still seems like it would provide better accuracy than just reusing the values 4 times.

@jgallagher59701 - your input is requested.

[captainkirk99]

James sez:

I’m not sure where I saw this, but there was a file (MOD09? - I think so) that had dimension variables (explicit ones) for each of three resolutions but the STAREmaster code was reading only one and using it to compute the values for the others. Why not read them? The computed values and the values of the dimension variables were not the same.

Unless I was hallucinating… ;-)

I’ll be back online on Tuesday.

James

[captainkirk99]

Well James, perhaps the high temps up there are causing hallucinations, because I don't see any lat/lons in the file except the 1km resolution...

[captainkirk99]

Also in the MOD03 geolocation file, I see that only 1 km resolution is provided:

[captainkirk99]

For reference, here's the sizes of the different resolution's arrays: dimensions: i_1km = 2030 ; j_1km = 1354 ; i_500m = 4060 ; j_500m = 2708 ; i_250m = 8120 ; j_250m = 5416 ;

[captainkirk99]

So we see that both MOD09 and MOD03 only provide 1km resolution lat/lon data, of array size 2030 x 1354.

[captainkirk99]

@michaelleerilee suggests the code in https://github.com/SpatioTemporal/GeoData/tree/master/geodata/modis_coarse_to_fine_geolocation

This is about 500 lines of python code which does the interpolation. So, is the answer here to re-implement this in C++ and then interpolate the 1km lat/lon data to 250m and 500m?

[jgallagher59701]

I looked at the mod09 data and saw that there were explicit lat/lon data for each of the resolutions. I will look into it more today and get back to you.

[captainkirk99]

OK if you can point it out, I am happy to use it...

Perhaps you are looking at a netCDF file or some other version of the data? I am looking at the HDF4 file, but can use the netCDF file if that has the data...

[jgallagher59701]

Here's a MOD09 file served by Hyrax: http://test.opendap.org/opendap/STARE/MYD09.A2019003.2040.006.2019005020913.hdf.html

The domain variables (Latitude, ...) are at the bottom.

I used constraints to access them and here are a subset of their values:

Dataset: MYD09.A2019003.2040.006.2019005020913.hdf Latitude[0], 32.2761, 32.274, 32.2719, 32.2697, 32.2676, 32.2654, 32.2633, 32.2612, 32.259, 32.2568 Latitude[1], 32.2942, 32.2921, 32.2899, 32.2877, 32.2855, 32.2833, 32.2811, 32.2789, 32.2766, 32.2745 Latitude[2], 32.3123, 32.3101, 32.3078, 32.3056, 32.3033, 32.3011, 32.2988, 32.2966, 32.2943, 32.2921 Latitude[3], 32.3304, 32.3281, 32.3258, 32.3235, 32.3212, 32.3189, 32.3166, 32.3143, 32.312, 32.3097 Latitude[4], 32.3485, 32.3462, 32.3438, 32.3415, 32.3391, 32.3367, 32.3344, 32.332, 32.3297, 32.3273 Latitude[5], 32.3666, 32.3642, 32.3618, 32.3594, 32.3569, 32.3545, 32.3521, 32.3497, 32.3473, 32.345 Latitude[6], 32.3847, 32.3823, 32.3798, 32.3773, 32.3748, 32.3724, 32.3699, 32.3675, 32.365, 32.3626 Latitude[7], 32.4028, 32.4003, 32.3978, 32.3952, 32.3927, 32.3902, 32.3877, 32.3852, 32.3827, 32.3802 Latitude[8], 32.4209, 32.4184, 32.4158, 32.4132, 32.4106, 32.408, 32.4055, 32.4029, 32.4004, 32.3978 Latitude[9], 32.439, 32.4364, 32.4338, 32.4311, 32.4285, 32.4259, 32.4233, 32.4206, 32.418, 32.4154 Latitude_1[0], 32.2761, 32.2751, 32.274, 32.2729, 32.2719, 32.2708, 32.2697, 32.2687, 32.2676, 32.2665 Latitude_1[1], 32.2852, 32.2841, 32.283, 32.282, 32.2809, 32.2798, 32.2787, 32.2776, 32.2765, 32.2754 Latitude_1[2], 32.2942, 32.2931, 32.2921, 32.291, 32.2899, 32.2888, 32.2877, 32.2866, 32.2855, 32.2844 Latitude_1[3], 32.3033, 32.3022, 32.3011, 32.3, 32.2989, 32.2977, 32.2966, 32.2955, 32.2944, 32.2933 Latitude_1[4], 32.3123, 32.3112, 32.3101, 32.309, 32.3078, 32.3067, 32.3056, 32.3045, 32.3033, 32.3022 Latitude_1[5], 32.3214, 32.3202, 32.3191, 32.318, 32.3168, 32.3157, 32.3146, 32.3134, 32.3123, 32.3111 Latitude_1[6], 32.3304, 32.3293, 32.3281, 32.327, 32.3258, 32.3247, 32.3235, 32.3224, 32.3212, 32.3201 Latitude_1[7], 32.3395, 32.3383, 32.3372, 32.336, 32.3348, 32.3337, 32.3325, 32.3313, 32.3301, 32.329 Latitude_1[8], 32.3485, 32.3474, 32.3462, 32.345, 32.3438, 32.3426, 32.3415, 32.3403, 32.3391, 32.3379 Latitude_1[9], 32.3576, 32.3564, 32.3552, 32.354, 32.3528, 32.3516, 32.3504, 32.3492, 32.348, 32.3468 Latitude_2[0], 32.2716, 32.2711, 32.2705, 32.27, 32.2695, 32.269, 32.2684, 32.2679, 32.2674, 32.2669 Latitude_2[1], 32.2761, 32.2756, 32.2751, 32.2745, 32.274, 32.2735, 32.2729, 32.2724, 32.2719, 32.2713 Latitude_2[2], 32.2806, 32.2801, 32.2796, 32.2791, 32.2785, 32.278, 32.2774, 32.2769, 32.2764, 32.2758 Latitude_2[3], 32.2852, 32.2846, 32.2841, 32.2836, 32.283, 32.2825, 32.282, 32.2814, 32.2809, 32.2803 Latitude_2[4], 32.2897, 32.2892, 32.2886, 32.2881, 32.2875, 32.287, 32.2865, 32.2859, 32.2854, 32.2848 Latitude_2[5], 32.2942, 32.2937, 32.2931, 32.2926, 32.2921, 32.2915, 32.291, 32.2904, 32.2899, 32.2893 Latitude_2[6], 32.2987, 32.2982, 32.2977, 32.2971, 32.2966, 32.296, 32.2955, 32.2949, 32.2944, 32.2938 Latitude_2[7], 32.3033, 32.3027, 32.3022, 32.3016, 32.3011, 32.3005, 32.3, 32.2994, 32.2989, 32.2983 Latitude_2[8], 32.3078, 32.3072, 32.3067, 32.3061, 32.3056, 32.305, 32.3045, 32.3039, 32.3034, 32.3028 Latitude_2[9], 32.3123, 32.3118, 32.3112, 32.3107, 32.3101, 32.3095, 32.309, 32.3084, 32.3078, 32.3073

Here's the URL to get that: http://test.opendap.org/opendap/STARE/MYD09.A2019003.2040.006.2019005020913.hdf.ascii?Latitude%5B0:9%5D%5B0:9%5D,Latitude_1%5B0:9%5D%5B0:9%5D,Latitude_2%5B0:9%5D%5B0:9%5D

And a more readable version: http://test.opendap.org/opendap/STARE/MYD09.A2019003.2040.006.2019005020913.hdf.ascii?Latitude[0:9][0:9],Latitude_1[0:9][0:9],Latitude_2[0:9][0:9]

It is possible that the server is adding the two higher res domain information, so let me check...

[jgallagher59701]

BTW, this is what I mean by 'We could read the values with opendap.' The same access works regardless of file type, etc. and the server supports binary access. I'm using ascii here because it's easy to read ;-)

[jgallagher59701]

OK, looking at this file with HDFView, I can see the Latitude_1, ..., etc., domain variables are generated by the server. So, I guess we need to interpolate the values. Arrgh. Sorry for the confusion.

[jgallagher59701]

Yes. Thanks and sorry for the confusion.

[captainkirk99]

OK, does that mean you do want me to translate the interpolation code into C++ and build it into staremaster?

Do you want me to take code from bes for this? Or start with Michael's python? Either way is fine by me...

On Fri, Jul 23, 2021 at 2:09 PM James Gallagher @.***> wrote:

Yes. Thanks and sorry for the confusion.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SpatioTemporal/STAREmaster/issues/188#issuecomment-885887652, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI4M44PWGOFLJ347RGTRXB3TZHD6HANCNFSM5AND5RDA .